XT2 Snapshot 20240910

Users
,
1

Hey folks - another week, another snapshot :rocket:

Firstly, let’s talk breaking changes: with this snapshot, we’re taking advantage of our short time remaining in ‘early access’ to release a number of outstanding breaking changes to the tx-log and object-store file formats in one big batch.

Most notably, we’ve taken on board your feedback on the column ‘normalisation’ - while we hoped this would be useful for users using XT2 as an OLTP database from Clojure and OLAP using their existing Clojure tools, this turned out to be more confusing and complex (and hence, buggy) than it was worth in practice. We’ve cut this back significantly, particularly from the SQL interface - we didn’t have support for double-quoted “delimited identifiers” for quite a while, which are much more useful in polyglot environments. For more details, see #3675.

Under the hood, we’ve also taken the opportunity of the breaking change to introduce the file-format changes required for ‘schemas’ (as in namespaces for your tables, rather than type-checking). At the moment, we have four: public (where all your tables go), xt (where all our tables go), information_schema (metadata about those tables), and pg_catalog (also metadata about those tables, but in a slightly different format) - but we also now have the potential to add functionality in this area later without requiring further breaking changes.

tl;dr is that you’ll need to reset your tx-logs and object-stores, I’m afraid. For now we’re assuming that, given we are still “early access”, that anyone for whom this is likely to be a significant issue is already in contact with us directly - you are, right?

Ok, that’s the breaking changes - we also have a couple of significant additions which are much more interesting :smiley:

  • In SQL, we’ve added the ability to add documents directly into tables, using INSERT INTO my_table RECORDS ?, {_id: 1, a: ?} - you can either write literal objects in your SQL statements, or pass the whole document map as a parameter.
  • Related: as well as Postgres-compatible JSON support, we’ve added support for a custom Postgres type - Transit. This’ll allow you to preserve types that aren’t naturally available through vanilla JSON (i.e. no more string timestamps!).
  • Initial support for the ROW_NUMBER window function. This is largely for Postgres wire-compatibility (a number of Postgres drivers make queries involving ROW_NUMBER to retrieve database metadata, sometimes on startup), but paves the way for the remaining window functions to be added soon.
  • We’ve also added generate_series, again from Postgres.
  • Previously, we required that transactions through the Postgres-compatible server were started with an explicit BEGIN READ ONLY/BEGIN READ WRITE. We now infer this from the first statement within the transaction, so you can open transactions in the usual way for your driver.
  • All the other changes and bugfixes are listed in the milestone.

Safe to say, it’s been a busy week! (well, 8 days, but who’s really counting?)

In other news: next up on our list is to release our first ‘beta’ of XT2 (we’ve been in “early access” for so long it probably makes sense to skip ‘alpha’). We’re really excited to be in touching distance of this milestone, and we couldn’t have done it without all of your feedback and help tracking down issues - thank you!

As always, let us know if you run into any issues, or have any questions - cheers!

XT Team

1 Like