We’re currently considering whether we want to use our XTDB cluster to also store time-series data. Does anyone have any experience around this?
I’ve also read in an old Github issue that there were some plans for turning off transactions for time-series data. Is this still on the roadmap for XTDB?
1 Like
Hey @xpekeler! “Time series” usually comes loaded with expectations around: high-volume (inc. sharding, columnar storage, roll-up retention, partitioning over time), non-transactional (i.e. no need for ACID or integrity constraints…which is the tradeoff for being high-volume), and streaming aggregation support. Unfortunately XT doesn’t currently cater for these areas particularly well, although we are definitely aiming to tackle complex aggregation support as a reasonably high priority in a future major release.
To be more explicit about aggregations, querying across entity histories currently means doing a lot of manual work outside of the Datalog (since the Datalog engine is designed to work at a single point-in-time), e.g. see these benchmark tests we adapted from Timescale https://github.com/xtdb/xtdb/blob/e2f51ed99fc2716faa8ad254c0b18166c937b134/bench/src/xtdb/bench/ts_devices.clj and https://github.com/xtdb/xtdb/blob/e2f51ed99fc2716faa8ad254c0b18166c937b134/bench/src/xtdb/bench/ts_weather.clj
All that said, XT may still be an acceptable fit for certain situations with modest requirements, particularly where owning and operating additional time-series-specific infrastructure is more effort than ‘just’ manually crafting a few queries on top of what is already being deployed.
I’ve also read in an old Github issue that there were some plans for turning off transactions for time-series data. Is this still on the roadmap for XTDB?
I can’t recall exactly which issue you are referencing, but if I understand correctly, you’re asking about some kind of specific configuration for non-transactional workloads, where transactions are no longer ACID in order to maximise ingestion performance. (Please correct me if that’s wrong!). I can think of a few areas of low hanging fruit that could help the situation, but we don’t have any active roadmap plans in this area currently. Feel free to comment on any issues you find interesting though, and continue asking as many questions as you like here 
2 Likes