[2.x] How Filter Query Results Based On Nested Vector Values?

Users
1

Hello! Been scouring github (implementation, tests, and issues/discussion) as well as stack overflow and this site and haven’t been able to find a answer on my following problem.

I have the following data in a in-memory node:

(ns user
  (:require
     [xtdb.api :as xt]
     [xtdb.node :as xt.node]))

(def n (xt.node/start-node {}))

(xt/submit-tx n [[:put :store {:xt/id 0
                               :name "Some Store"
                               :inventory
                               [{:display "Cool shirt"
                                 :price   10.99
                                 :tags    [:cool :shirt :trending]}]}]])

I am then attempting to query/filter results based off a certain tag being present in the list/collection. With my current mental model, I am under the impression that the inventory vector will be “unwrapped”, and subsequently the tags vector will also be unwrapped and checked in order to filter the results.

(xt/q n '{:find  [display price store-id]
          :where [($ :store [{:xt/id store-id, :inventory [inventory ...]}])
                  [(. inventory :display) display]
                  [(. inventory :price) price]
                  [(. inventory :tags) [tags ...]]
                  [(= tags :trending)]]})

I have attempted to normalize the tag data into its own table, like so:

(xt/submit-tx n [[:put :store {:xt/id 0
                               :name "Some Store"
                               :inventory
                               [{:display "Some cool shirt"
                                 :price   10.99
                                 :tag-ids #{0 1 2}}]}]
                 [:put :tags {:xt/id 0, :tag :trending}]
                 [:put :tags {:xt/id 1, :tag :cool}]
                 [:put :tags {:xt/id 2, :tag :shirt}]])

But even with the above, I am not sure how to query based on the tag-ids set. I have also attempted to make the :tag-ids field a vector to no avail.

At this point, I’m pretty confident my mental model of xtdb datalog isn’t correct, which is leading me to bash my head against a wall.

I understand the the data models are not totally normalized, I am mainly attempting to exercise and wrap my head around what is possible with xtdb. If the above example is possible to perform, how should a query be constructed? If not, any suggestions on how to model the data?

Any help with correcting my mental model as well is greatly appreciated :slightly_smiling_face:

Finally, I haven’t had a long time to write this question out and proof all the tx and q operations, but I think they catch the gist of my meaning. If not, and there isn’t clarity, I am happy to clarify!

Also, I came across this github issue, which seems to be related to querying/filtering based off nested vectors. My attempt to translate the example in the github issue to the example data I’ve provided should be:

(xt/q n '{:find [store-id item]
          :where [($ :stores [{:xt/id store-id} inventory])
                  ($ [inventory {:tags tags}]) 
                  ($ [tags {:as tag}])
                  [(= tags :trending)]]})

However, this gives a “malformed query” error.

2

Hey there @bnert, thanks for giving v2 a try!

I’m afraid this isn’t yet fully supported - support for nested/composite data structures is something we’re looking to extend in the run up to our first alpha release later this year.

If you’d be interested, we’re working closely with a number of keen early adopters to elicit feedback (and the inevitable issue reports!) - let me know and I can add you to the list :slight_smile:

James

3

Thanks for taking the time to respond, @jarohen!

That makes sense. I can work around the current limitation then :slight_smile:

If you’d be interested, we’re working closely with a number of keen early adopters to elicit feedback (and the inevitable issue reports!) - let me know and I can add you to the list :slight_smile:

Sure! I’d be happy to contribute! I’ll be on the lookout for a DM.

2 Likes
4

I’ll be on the lookout for a DM.

Excellent! I shall handle it from here :slight_smile:

1 Like
5

A follow up to this, I have found a work around.

Instead of using a set (i.e. #{:a :b :c}), a map which maps to true values can be used (i.e. {:a true, :b true, :c true}.

The data model would look like:

{:xt/id 0
 :name "Some Store"
 :inventory
 [{:display "Cool shirt"
   :price 10.99
   :tags {:cool true, :shirt true, :trending true}}]}

There resulting query to find the “trending” tag would be:

(xt/q n
      '{:find   [display price store-id]
         :where [($ :stores [{:xt/id store-id :inventory [item ...]}])
                 [(. item :display) display]
                 [(. item :price) price]
                 [(.. item :tags :trending) trending?]
                 [(true? trending?)]]})
1 Like