Previously, SQL operations returned :xt$id and :first_name etc, following allowed SQL naming.
The latest snapshot seems to have changed this to return :xt/id and :first-name – presumably to match what XTQL operations return.
Unfortunately, that means you can’t pass that data back through either HoneySQL or next.jdbc because those libraries have a long-standing convention that :foo/bar means table foo, column bar.
You can still pass in the hyphenated names as long as you enable ANSI quoting (which was not required before this change).
Is this change just a temporary bug, or are we stuck with this now?
You should be able to achieve this behaviour using the :key-fn option passed to xt/q - passing :snake-case-string should preserve the SQL naming. You can then keywordise separately as required - our :snake-case-keyword is what interprets $ as namespace separators.
Obviously not our intention to make XT unusable from next.jdbc with any of these changes, please do continue letting us know Similarly, if there’s any smoke tests with next.jdbc.xt that we can easily run as part of our pipeline, that would help us spot these things earlier too.
OK, I’ve updated next.jdbc.xt to pass :key-fn :snake-case-string and then call clojure.walk/keywordize-keys on the result and that works great with an in-memory node.
However, it doesn’t look like the Docker image ghcr.io/xtdb/xtdb-standalone-ea has been updated yet to be compatible with this?
When I try to run the usermanager example with the latest next.jdbc.xt, the tests work with an in-memory node but fail with a client node against the latest Docker version with this error:
IllegalArgumentException(key=:unknown-deserialization-opt, message=Illegal argument: 'unknown-deserialization-opt', data={:key-fn :snake-case-string}, cause=null)
at xtdb.error$illegal_arg.invokeStatic(error.clj:11)
at xtdb.error$illegal_arg.invoke(error.clj:3)
at xtdb.error$illegal_arg.invokeStatic(error.clj:7)
at xtdb.error$illegal_arg.invoke(error.clj:3)
at xtdb.client.impl$validate_remote_key_fn.invokeStatic(impl.clj:82)
at xtdb.client.impl$validate_remote_key_fn.invoke(impl.clj:80)
at xtdb.client.impl$validate_query_opts.invokeStatic(impl.clj:85)
at xtdb.client.impl$validate_query_opts.invoke(impl.clj:84)
at xtdb.client.impl.XtdbClient.open_query_AMPERSAND_(impl.clj:92)
Here’s my Docker commands & output:
sean@sean-win11-desk:~/oss/usermanager-example$ docker pull ghcr.io/xtdb/xtdb-standalone-ea
Using default tag: latest
latest: Pulling from xtdb/xtdb-standalone-ea
...
Digest: sha256:299de4cc84236d092d29bc9fa3ee3b934cadc542d8a6aafcee52f0a0f34b33e5
Status: Downloaded newer image for ghcr.io/xtdb/xtdb-standalone-ea:latest
ghcr.io/xtdb/xtdb-standalone-ea:latest
What's Next?
View a summary of image vulnerabilities and recommendations → docker scout quickview ghcr.io/xtdb/xtdb-standalone-ea
sean@sean-win11-desk:~/oss/usermanager-example$ docker run -tip 3000:3000 ghcr.io/xtdb/xtdb-standalone-ea
Starting XTDB 2.x (pre-alpha) @ "dev-SNAPSHOT" @ d508418 ...
18:47:15 | INFO xtdb.server | HTTP server started on port: 3000
18:47:15 | INFO xtdb.cli | Node started
Hi @seancorfield apologies for the confusion - we don’t have automated dev builds being published quite yet (to provide some stability for pre-alpah users) - but the new Docker latest image is now live: xtdb-standalone-ea versions · xtdb · GitHub
docker run -tip 3000:3000 ghcr.io/xtdb/xtdb-standalone-ea
Starting XTDB 2.x (pre-alpha) @ "dev-SNAPSHOT" @ e294fd9 ...
18:01:16 | DEBUG xtdb.metrics | Metrics server started on port: 8080
18:01:17 | INFO xtdb.server | HTTP server started on port: 3000
18:01:17 | INFO xtdb.cli | Node started
So that is current as of 3 days ago.
But when I run my usermanager app I still get this exception:
IllegalArgumentException(key=:unknown-deserialization-opt, message=Illegal argument: 'unknown-deserialization-opt', data={:key-fn :snake-case-string}, cause=null)
at xtdb.error$illegal_arg.invokeStatic(error.clj:11)
at xtdb.error$illegal_arg.invoke(error.clj:3)
at xtdb.error$illegal_arg.invokeStatic(error.clj:7)
at xtdb.error$illegal_arg.invoke(error.clj:3)
at xtdb.client.impl$validate_remote_key_fn.invokeStatic(impl.clj:82)
at xtdb.client.impl$validate_remote_key_fn.invoke(impl.clj:80)
at xtdb.client.impl$validate_query_opts.invokeStatic(impl.clj:85)
at xtdb.client.impl$validate_query_opts.invoke(impl.clj:84)
at xtdb.client.impl.XtdbClient.open_query_AMPERSAND_(impl.clj:92)
at xtdb.api$q_AMPERSAND_.invokeStatic(api.clj:106)
at xtdb.api$q_AMPERSAND_.invoke(api.clj:55)
at xtdb.api$q.invokeStatic(api.clj:162)
at xtdb.api$q.invoke(api.clj:114)
at next.jdbc.xt$_execute_all_STAR_.invokeStatic(xt.clj:41)
at next.jdbc.xt$_execute_all_STAR_.invoke(xt.clj:39)
at next.jdbc.xt$fn__9191.invokeStatic(xt.clj:114)
at next.jdbc.xt$fn__9191.invoke(xt.clj:107)
at next.jdbc.protocols$fn__5246$G__5226__5255.invoke(protocols.clj:34)
at next.jdbc$execute_BANG_.invokeStatic(jdbc.clj:257)
at next.jdbc$execute_BANG_.invoke(jdbc.clj:241)
at next.jdbc.sql$query.invokeStatic(sql.clj:106)
at next.jdbc.sql$query.invoke(sql.clj:97)
at next.jdbc.sql$query.invokeStatic(sql.clj:103)
at next.jdbc.sql$query.invoke(sql.clj:97)
at usermanager.model.user_manager$get_users.invokeStatic(user_manager.clj:90)
at usermanager.model.user_manager$get_users.invoke(user_manager.clj:87)
Ah, it looks like you might be picking up a stale client dependency still (because the validate-remote-key-fn seen in the stacktrace no longer exists) - can you check that you have the latest SNAPSHOT versions on your classpath? The following coordinates should resolve to the versions around Mon Feb 26 10:28:01 UTC 2024here
;; xtdb-api for the main public API, for both remote-client and in-process nodes
com.xtdb/xtdb-api {:mvn/version "2.0.0-SNAPSHOT"}
;; xtdb-http-client-jvm for connecting to a remote server
com.xtdb/xtdb-http-client-jvm {:mvn/version "2.0.0-SNAPSHOT"}
Figured out part of the problem: I still referred to xtdb-http-client-clj instead of xtdb-http-client-jvm – updating that got me a little further.
Now I have the following error [solved – see further down the thread]:
(~/oss/usermanager-example)-(!2021)-> rm -rf target/
Mon Feb 26 11:37:05
(~/oss/usermanager-example)-(!2022)-> rm -rf ~/.m2/repository/com/xtdb/
Mon Feb 26 11:37:08
(~/oss/usermanager-example)-(!2023)-> rm -rf .cpcache/
Mon Feb 26 11:37:11
(~/oss/usermanager-example)-(!2024)-> clojure -T:build ci
Downloading: com/xtdb/xtdb-api/2.0.0-SNAPSHOT/maven-metadata.xml from ossrh-snapshots
Downloading: com/xtdb/xtdb-http-client-jvm/2.0.0-SNAPSHOT/maven-metadata.xml from ossrh-snapshots
Downloading: com/xtdb/xtdb-core/2.0.0-SNAPSHOT/maven-metadata.xml from ossrh-snapshots
Downloading: com/xtdb/xtdb-http-client-jvm/2.0.0-SNAPSHOT/xtdb-http-client-jvm-2.0.0-20240226.102659-3.pom from ossrh-snapshots
Downloading: com/xtdb/xtdb-api/2.0.0-SNAPSHOT/xtdb-api-2.0.0-20240226.102659-7.pom from ossrh-snapshots
Downloading: com/xtdb/xtdb-core/2.0.0-SNAPSHOT/xtdb-core-2.0.0-20240226.102659-7.pom from ossrh-snapshots
Downloading: com/xtdb/xtdb-http-client-jvm/2.0.0-SNAPSHOT/xtdb-http-client-jvm-2.0.0-20240226.102659-3.jar from ossrh-snapshots
Downloading: com/xtdb/xtdb-core/2.0.0-SNAPSHOT/xtdb-core-2.0.0-20240226.102659-7.jar from ossrh-snapshots
Downloading: com/xtdb/xtdb-api/2.0.0-SNAPSHOT/xtdb-api-2.0.0-20240226.102659-7.jar from ossrh-snapshots
...
(tests with in-memory node pass, JAR gets built)
...
But when I try to run it:
(~/oss/usermanager-example)-(!2025)-> java -jar target/usermanager/example-standalone.jar
...
Caused by: java.lang.IllegalStateException: Attempting to call unbound fn: #'xtdb.serde/read-key-fn
at clojure.lang.Var$Unbound.throwArity(Var.java:45)
at clojure.lang.AFn.invoke(AFn.java:32)
at clojure.lang.Var.invoke(Var.java:384)
at clojure.lang.LispReader$CtorReader.readTagged(LispReader.java:1448)
at clojure.lang.LispReader$CtorReader.invoke(LispReader.java:1427)
at clojure.lang.LispReader$DispatchReader.invoke(LispReader.java:846)
at clojure.lang.LispReader.read(LispReader.java:285)
at clojure.lang.LispReader.read(LispReader.java:216)
at clojure.lang.LispReader.read(LispReader.java:205)
at clojure.lang.RT.readString(RT.java:1876)
at clojure.lang.RT.readString(RT.java:1871)
at xtdb.client.impl.XtdbClient.<clinit>(impl.clj:91)
What am I missing now? [solved – see further down the thread]
I think this might be an AOT-related issue. I can run clojure -M -m usermanager.main but not the uberjar.
In next.jdbc.xt, I have macros to extend the next.jdbc protocols to whatever XT types are on the classpath – either the Node impl or the XtdbClient impl – and it does that by attempting to require the .impl ns that defines those types (and then using Class/forName to load the type that will be extended to).
Similarly, if there’s any smoke tests with