XTDB 2 local storage

I’ve been peeking in at the development of version 2 for a while and since the last time I looked a lot has changed!

I tried to find a filesystem based object storage module, but I don’t see one in the modules folder, is something like that possible now, or is it planned?

I’m using version 1 as a supplementary DB right now and would love to investigate building an ActiveRecord driver for Ruby on Rails for XTDB 2.

1 Like

Hey @caleb_land the v2 configuration APIs have changed quite a lot in the past couple of weeks. You can see a preview of the see the latest instructions here (for Clojure): Configuration Cookbook | XTDB and for Rubyists we’ll have a new config.yaml mechanism (e.g. S3 | XTDB) - but note we’ve not updated the Docker/Maven snapshots to reflect the latest version on the 2.x branch yet. That should happen within the next week though.

However, in any case there is no explicit “filesystem” object storage - it is essentially just the buffer pool cache directory (in a non-clustered setup) - and you can see an example of how/where that gets configured (prior to our latest changes) here xtdb/docker/xtdb.edn at 6ad8d7fa4adadcb486732a8634b073501bf1397a · xtdb/xtdb · GitHub (i.e. you should be able to mount a docker volume at the path shown to achieve what you’re wanting)

Thanks for the update, I’ll check it out and start playing around.

Running XTDB 2 in a non-clustered setup shouldn’t be any crazier than running a single node PostgreSQL setup from an architectural perspective (with the understanding that 2 is pre-alpha)?


Yep you could definitely view a single-node XT2 and single-node Postgres as sharing the same durability and (low) availability guarantees. Postgres offers explicit and documented WAL shipping though, so the migration to a clustered setup is more straightforward. With a single-node XT2 currently you would probably need our help to seamlessly switch over to Kafka and introduce clustering (something we’ll solve properly in due course), but for now you can always manually export & re-import.

1 Like