I would like to track changes to records based on whether there’s a history entry.
If there’s a schema change, however, it still records a history entry, as you would expect.
I’m looking for advice on how others have handled this particular issue - whether by building abstractions or by convention.
This other topic touches on it, however I’m not looking for future use, I need to come up with a solution in the short term.
My current thought is to store a key for the type of data change, and query that key as a matter of convention going forward. As such, I have a question about the internal performance of querying - does it retrieve the entire document if I’m only querying one key?
Off the top of my head there are a few options (in no particular order):
Version your tables, e.g. such that you have products1 and later products2 when the schema needs to change - this approach would work better once XT2 supports VIEWs (which can help with avoiding pushing too much of that complexity into downstream queries)
Use new IDs and capture the schema version as an additional key(/column/attribute) and filter on it. However, this feels less than ideal from a modelling and performance perspective
Version all individual keys(/columns/attributes) and track the set of ‘current’ keys for the current schema separately. e.g. given product_name1, product_name2, and product_tax4, the current schema might only be product_name2 and product_tax4
No, XTDB should be able to accelerate all queries against top-level keys(/columns/attributes) - and this is true of both v1 and v2 (although the underlying mechanics are quite different!)
It might be simpler to approach this topic from the perspective of the requirements of your queries - can you say/illustrate more about what you’re needing?
