String function fails when column contain NULLs [solved]

Users
1

Hello,

When I put a document with a username field for example:

(xt/submit-tx
    xt-node
    [[:put-docs :users {:xt/id    (random-uuid)
                        :username "Bob"}]])

then query using String function lower:

(xt/q
    xt-node
    '(-> (from :users [username])
         (where (= (lower username) "bob"))))

it returns the result as expected:

[{:username "Bob"}]

but if we put a namespaced field my/username:

(xt/submit-tx
    xt-node
    [[:put-docs :users {:xt/id    (random-uuid)
                        :my/username "Bob"}]])

then query using String function lower:

(xt/q
    xt-node
    '(-> (from :users [my/username])
         (where (= (lower my/username) "bob"))))

it throws an exception:

1. Unhandled java.lang.IllegalArgumentException
   No method in multimethod 'codegen-call' for dispatch value: [:lower
   :absent]

              MultiFn.java:  156  clojure.lang.MultiFn/getFn
              MultiFn.java:  229  clojure.lang.MultiFn/invoke
            expression.clj:  582  xtdb.expression/codegen-call*/iter/fn
              LazySeq.java:   42  clojure.lang.LazySeq/sval
              LazySeq.java:   51  clojure.lang.LazySeq/seq
                 Cons.java:   39  clojure.lang.Cons/next
                   RT.java:  713  clojure.lang.RT/next
                  core.clj:   64  clojure.core/next
             protocols.clj:  169  clojure.core.protocols/fn
             protocols.clj:  124  clojure.core.protocols/fn
             protocols.clj:  692  clojure.core.protocols/fn/G
             protocols.clj:   31  clojure.core.protocols/seq-reduce
             protocols.clj:   75  clojure.core.protocols/fn
             protocols.clj:   75  clojure.core.protocols/fn
             protocols.clj:  692  clojure.core.protocols/fn/G
                  core.clj: 6886  clojure.core/reduce
                  core.clj: 6958  clojure.core/into
                  core.clj: 6950  clojure.core/into
            expression.clj:  578  xtdb.expression/codegen-call*
            expression.clj:  568  xtdb.expression/codegen-call*
            expression.clj:  631  xtdb.expression/eval80969/fn
              MultiFn.java:  234  clojure.lang.MultiFn/invoke
            expression.clj:  625  xtdb.expression/eval80969/fn/iter/fn/fn
            expression.clj:  624  xtdb.expression/eval80969/fn/iter/fn
              LazySeq.java:   42  clojure.lang.LazySeq/sval
              LazySeq.java:   51  clojure.lang.LazySeq/seq
                   RT.java:  535  clojure.lang.RT/seq
                  core.clj:  139  clojure.core/seq
             protocols.clj:   24  clojure.core.protocols/seq-reduce
             protocols.clj:   75  clojure.core.protocols/fn
             protocols.clj:   75  clojure.core.protocols/fn
             protocols.clj:  692  clojure.core.protocols/fn/G
                  core.clj: 6886  clojure.core/reduce
                  core.clj: 6868  clojure.core/reduce
            expression.clj:  571  xtdb.expression/codegen-call*
            expression.clj:  568  xtdb.expression/codegen-call*
            expression.clj:  631  xtdb.expression/eval80969/fn
              MultiFn.java:  234  clojure.lang.MultiFn/invoke
            expression.clj:  625  xtdb.expression/eval80969/fn/iter/fn/fn
            expression.clj:  624  xtdb.expression/eval80969/fn/iter/fn
              LazySeq.java:   42  clojure.lang.LazySeq/sval
              LazySeq.java:   51  clojure.lang.LazySeq/seq
                   RT.java:  535  clojure.lang.RT/seq
                  core.clj:  139  clojure.core/seq
             protocols.clj:   24  clojure.core.protocols/seq-reduce
             protocols.clj:   75  clojure.core.protocols/fn
             protocols.clj:   75  clojure.core.protocols/fn
             protocols.clj:  692  clojure.core.protocols/fn/G
                  core.clj: 6886  clojure.core/reduce
                  core.clj: 6868  clojure.core/reduce
            expression.clj:  571  xtdb.expression/codegen-call*
            expression.clj:  568  xtdb.expression/codegen-call*
            expression.clj:  631  xtdb.expression/eval80969/fn
              MultiFn.java:  234  clojure.lang.MultiFn/invoke
            expression.clj: 1484  xtdb.expression/fn
            expression.clj: 1481  xtdb.expression/fn
                  util.clj:  520  xtdb.util/lru-memoize/fn
            expression.clj: 1474  xtdb.expression/wrap-zone-id-cache-buster/fn
            expression.clj: 1522  xtdb.expression/->expression-projection-spec
            expression.clj: 1520  xtdb.expression/->expression-projection-spec
            expression.clj: 1550  xtdb.expression/->expression-relation-selector
            expression.clj: 1549  xtdb.expression/->expression-relation-selector
                  scan.clj:  435  xtdb.operator.scan/eval90872/fn/reify/iter/fn
              LazySeq.java:   42  clojure.lang.LazySeq/sval
              LazySeq.java:   51  clojure.lang.LazySeq/seq
                   RT.java:  535  clojure.lang.RT/seq
                  core.clj:  139  clojure.core/seq
             protocols.clj:   24  clojure.core.protocols/seq-reduce
             protocols.clj:   75  clojure.core.protocols/fn
             protocols.clj:   75  clojure.core.protocols/fn
             protocols.clj:  692  clojure.core.protocols/fn/G
                  core.clj: 6886  clojure.core/reduce
                  core.clj: 6958  clojure.core/into
                  core.clj: 6950  clojure.core/into
                  scan.clj:  430  xtdb.operator.scan/eval90872/fn/reify
                  scan.clj:  493  xtdb.operator.scan/eval91013/fn
              MultiFn.java:  234  clojure.lang.MultiFn/invoke
                 query.clj:  124  xtdb.query/prepare-ra/reify/reify
    ConcurrentHashMap.java: 1708  java.util.concurrent.ConcurrentHashMap/computeIfAbsent
                 query.clj:  112  xtdb.query/prepare-ra/reify
                 query.clj:  216  xtdb.query/open-query
                 query.clj:  211  xtdb.query/open-query
                  impl.clj:  114  xtdb.node.impl.Node/fn
                  util.clj:  276  xtdb.util//reify
    CompletableFuture.java:  684  java.util.concurrent.CompletableFuture/uniApplyNow
    CompletableFuture.java:  662  java.util.concurrent.CompletableFuture/uniApplyStage
    CompletableFuture.java: 2168  java.util.concurrent.CompletableFuture/thenApply
                  util.clj:  281  xtdb.util/then-apply
                  util.clj:  278  xtdb.util/then-apply
                  impl.clj:  103  xtdb.node.impl.Node/openQueryAsync
                   api.clj:  188  xtdb.api/eval27439/fn
             protocols.clj:  692  xtdb.protocols/eval26038/fn/G
                   api.clj:  106  xtdb.api/q&
                   api.clj:   55  xtdb.api/q&
                   api.clj:   99  xtdb.api/q&
                   api.clj:   55  xtdb.api/q&
                   api.clj:  158  xtdb.api/q
                   api.clj:  114  xtdb.api/q

subsequently, the same exception happens when re-trying the previous query without containing the namespace.

It seems like as soon as there is a namespaced field in the table, the String function(s) fail.

2

Hey @wkok I just updated the title to better reflect what I believe the core issue is here. Essentially NULL handling is not yet comprehensive across the execution engine. Related issues include:

and

We will definitely be closing this gap soon, in the meantime though whenever a NULL inadvertently creeps in during your workflow you will have to use different columns/tables or restart (drop) the database to get the queries executing again. Sorry about that!

3

Can confirm that starting with a blank database, putting the namespaced version of my document in first, the lower String function query works, and then breaks later when putting a document that would cause a NULL to be present, as you described.

Thanks, your explanation matches what Iā€™m seeing.

1 Like
4

Can confirm this is now working in the latest snapshot :+1:

Thanks!

1 Like