XTDB 2: unexpected server error compiling query

I start up a fresh XTDB Docker instance locally:

Status: Image is up to date for ghcr.io/xtdb/xtdb:latest
Starting XTDB 2.x @ 2.0.0-beta2 @ 2dc57ba ...
01:03:47 | INFO  xtdb.api.metrics.PrometheusMetrics | Prometheus server started on port 8080
01:03:47 | INFO  xtdb.metrics | tagging all metrics with node-id: xtdb-node-685c4c
01:03:47 | INFO  xtdb.pgwire | Server started on port: 5432
01:03:49 | INFO  xtdb.server | HTTP server started on port:  3000
01:03:49 | INFO  xtdb.cli | Node started

I run the following in a REPL, using next.jdbc and the PostgreSQL JDBC driver:

(~/oss/next-jdbc)-(!2005)-> clj -A:1.12:test
Clojure 1.12.0
user=> (require '[next.jdbc :as jdbc])
nil
user=> (require '[next.jdbc.sql :as sql])
nil
user=> (def db (jdbc/get-connection {:dbtype "postgres" :dbname "public"}))
SLF4J: No SLF4J providers were found.
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See https://www.slf4j.org/codes.html#noProviders for further details.
SLF4J: Class path contains SLF4J bindings targeting slf4j-api versions 1.7.x or earlier.
SLF4J: Ignoring binding found at [jar:file:/home/sean/.m2/repository/org/apache/logging/log4j/log4j-slf4j-impl/2.24.0/log4j-slf4j-impl-2.24.0.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See https://www.slf4j.org/codes.html#ignoredBindings for an explanation.
#'user/db
user=> (sql/query db ["select * from address"])
[]
user=> (jdbc/execute-one! db ["insert into address (_id,house,street,city,\"zip/code\",country) values (3, '5124', 'Ray Ave', 'Castro Valley', '94546', 'US')"])
#:next.jdbc{:update-count 0}
user=> (sql/query db ["select * from address"])
Execution error (PSQLException) at org.postgresql.core.v3.QueryExecutorImpl/receiveErrorResponse (QueryExecutorImpl.java:2733).
ERROR: unexpected server error compiling query

Here’s the console output from the Docker instance running:

01:04:21 | DEBUG xtdb.pgwire | Interpreting SQL:  select * from address
01:04:21 | WARN  xtdb.sql.plan | Table not found: address
01:04:33 | DEBUG xtdb.pgwire | Interpreting SQL:  insert into address (_id,house,street,city,"zip/code",country) values (3, '5124', 'Ray Ave', 'Castro Valley', '94546', 'US')
01:04:39 | DEBUG xtdb.pgwire | Interpreting SQL:  select * from address
01:04:39 | ERROR xtdb.pgwire | #error {
 :cause Cannot invoke "org.apache.arrow.vector.types.pojo.Field.getType()" because "field" is null
 :via
 [{:type java.lang.NullPointerException
   :message Cannot invoke "org.apache.arrow.vector.types.pojo.Field.getType()" because "field" is null
   :at [xtdb.types$field__GT_col_type invokeStatic types.clj 521]}]
 :trace
 [[xtdb.types$field__GT_col_type invokeStatic types.clj 521]
  [xtdb.types$field__GT_col_type invoke types.clj 520]
  [xtdb.operator.project$__GT_rename_projection_spec invokeStatic project.clj 109]
  [xtdb.operator.project$__GT_rename_projection_spec invoke project.clj 108]
  [xtdb.operator.project$eval24548$fn__24551$fn__24555$iter__24575__24579$fn__24580$fn__24581 invoke project.clj 162]
  [xtdb.operator.project$eval24548$fn__24551$fn__24555$iter__24575__24579$fn__24580 invoke project.clj 151]
  [clojure.lang.LazySeq force LazySeq.java 50]
  [clojure.lang.LazySeq lockAndForce LazySeq.java 60]
  [clojure.lang.LazySeq sval LazySeq.java 69]
  [clojure.lang.LazySeq unwrap LazySeq.java 77]
  [clojure.lang.LazySeq realize LazySeq.java 93]
  [clojure.lang.LazySeq seq LazySeq.java 106]
  [clojure.lang.RT seq RT.java 555]
  [clojure.core$seq__5484 invokeStatic core.clj 139]
  [clojure.core.protocols$seq_reduce invokeStatic protocols.clj 24]
  [clojure.core.protocols$fn__8258 invokeStatic protocols.clj 74]
  [clojure.core.protocols$fn__8258 invoke protocols.clj 74]
  [clojure.core.protocols$fn__8199$G__8194__8212 invoke protocols.clj 13]
  [clojure.core$transduce invokeStatic core.clj 7026]
  [clojure.core$into invokeStatic core.clj 7042]
  [clojure.core$into invoke core.clj 7029]
  [xtdb.operator.project$eval24548$fn__24551$fn__24555 invoke project.clj 170]
  [xtdb.logical_plan$unary_expr invokeStatic logical_plan.clj 116]
  [xtdb.logical_plan$unary_expr invoke logical_plan.clj 114]
  [xtdb.operator.project$eval24548$fn__24551 invoke project.clj 146]
  [clojure.lang.MultiFn invoke MultiFn.java 234]
  [xtdb.query$emit_expr$reify__27217 apply query.clj 128]
  [java.util.concurrent.ConcurrentHashMap computeIfAbsent ConcurrentHashMap.java 1708]
  [xtdb.query$emit_expr invokeStatic query.clj 116]
  [xtdb.query$emit_expr invoke query.clj 114]
  [xtdb.query$prepare_ra$reify__27256 columnFields query.clj 187]
  [xtdb.pgwire$parse$fn__28509 invoke pgwire.clj 1437]
  [xtdb.pgwire$parse invokeStatic pgwire.clj 1412]
  [xtdb.pgwire$parse invoke pgwire.clj 1383]
  [xtdb.pgwire$cmd_parse invokeStatic pgwire.clj 1464]
  [xtdb.pgwire$cmd_parse invoke pgwire.clj 1460]
  [xtdb.pgwire$handle_msg invokeStatic pgwire.clj 1622]
  [xtdb.pgwire$handle_msg invoke pgwire.clj 1600]
  [xtdb.pgwire$conn_loop invokeStatic pgwire.clj 1682]
  [xtdb.pgwire$conn_loop invoke pgwire.clj 1638]
  [xtdb.pgwire$connect invokeStatic pgwire.clj 1723]
  [xtdb.pgwire$connect invoke pgwire.clj 1691]
  [xtdb.pgwire$accept_loop$fn__28633$fn__28634$fn__28635 invoke pgwire.clj 1776]
  [clojure.lang.AFn run AFn.java 22]
  [java.util.concurrent.Executors$RunnableAdapter call Executors.java 572]
  [java.util.concurrent.FutureTask run FutureTask.java 317]
  [java.util.concurrent.ThreadPoolExecutor runWorker ThreadPoolExecutor.java 1144]
  [java.util.concurrent.ThreadPoolExecutor$Worker run ThreadPoolExecutor.java 642]
  [java.lang.Thread run Thread.java 1583]]}

It seems to be due to using "zip/code" as the column name – I was curious where we are now on / in column names since they used to be accepted and turned into $.

If I use "zip$code" it works (well, with a fresh instance since I can no longer query the address table in this instance :slight_smile: ).

Hey @seancorfield thanks for the bug report! I’ve done some initial triage and have opened an issue to track the resolution: Quoted columns containing '/' cause an NPE when being queried · Issue #3799 · xtdb/xtdb · GitHub

My understanding is that quoted column names like this should be able to round-trip fine, per PostgreSQL: Documentation: 7.0: Names

Current builds of XTDB disallow / in SQL entity names – either quoted or not – which I think is a perfectly reasonable solution, given that “qualified column names” is problematic for both HoneySQL and next.jdbc (I had added experimental support to both libraries for this but it was ugly to use and still had weird corner cases so I just removed it, in light of XTDB not accepting such names now).

FWIW, I have a branch of next.jdbc updated to run as many of its tests as possible against XTDB. I’ve reached 0 failures, 0 errors, mostly by updating all references to id to conditionally use _id, but also by disabling a number of tests. The required changes can be seen here: Comparing develop...xtdb-testing · seancorfield/next-jdbc · GitHub but if you run the test suite multiple times (NEXT_JDBC_TEST_XTDB=yes clojure -X:test), XTDB fails with an “unexpected error” on tx submit, so I’m going to leave the branch unmerged for now.

1 Like

As a follow-up to my next.jdbc / XTDB testing, this is the GH issue for the unexpected error on tx submit from multiple runs of next.jdbc test suite · Issue #3894 · xtdb/xtdb and I now have a PR open against next.jdbc to include XTDB as one of the “standard” databases for testing against in CI: enable xtdb testing automatically · seancorfield/next-jdbc@0282df7

I probably won’t merge that until #3894 is addressed, but I’ll try to keep the PR up-to-date so that any changes to next.jdbc also get tested against XTDB.

1 Like

Thanks for the update - we will prioritise accordingly (as far as I can tell this same issue could very easily crop up for others too!)

UPDATE: unexpected error on tx submit from multiple runs of next.jdbc test suite · Issue #3894 · xtdb/xtdb · GitHub is now resolved in main :tada: (but didn’t make the cut off for beta4, unfortunately!)

1 Like

Thank you.

And a new nightly build of the Docker image including that fix will be available… soon?

Yesterday’s nightly should already now be available and include the fix: xtdb versions · xtdb · GitHub (looks like you were ~3 hours too early for our scheduled job :slightly_smiling_face:)

Confirmed. I spun up next.jdbc’s tests this morning, got the very latest nightly Docker image this time, and ran the tests repeatedly with no issues. I’ll merge that branch in and XTDB will be part of next.jdbc’s standard test suite from now on!

1 Like