S3 errors for 2.0.0-beta6

I just tried 2.0.0-beta6, using the aws docker image.

Same everything, with the exception of the breaking changes of txLog as listed in the announcement.

Something odd is going on with the s3 connection. I’m using minio, hosted locally through a docker network.

To reiterate, it works with the previous version (I’m not sure which one it was, I was using :latest and I’ve already pruned the images)

Current errors:

Attaching to xtdb
xtdb  | Starting XTDB 2.x @ 2.0.0-beta6 @ df55068 ...
xtdb  | 08:32:42 | INFO  xtdb.metrics | tagging all metrics with node-id: 3f75e770
xtdb  | {:clojure.main/message
xtdb  |  "Execution error (NullPointerException) at xtdb.aws.S3$listAllObjects0$1/invokeSuspend (S3.kt:202).\nCannot invoke \"java.lang.Boolean.booleanValue()\" because the return value of \"software.amazon.awssdk.services.s3.model.ListObjectsV2Response.isTruncated()\" is null\n",
xtdb  |  :clojure.main/triage
xtdb  |  {:clojure.error/class java.lang.NullPointerException,
xtdb  |   :clojure.error/line 202,
xtdb  |   :clojure.error/cause
xtdb  |   "Cannot invoke \"java.lang.Boolean.booleanValue()\" because the return value of \"software.amazon.awssdk.services.s3.model.ListObjectsV2Response.isTruncated()\" is null",
xtdb  |   :clojure.error/symbol xtdb.aws.S3$listAllObjects0$1/invokeSuspend,
xtdb  |   :clojure.error/source "S3.kt",
xtdb  |   :clojure.error/phase :execution},
xtdb  |  :clojure.main/trace
xtdb  |  {:via
xtdb  |   [{:type java.lang.NullPointerException,
xtdb  |     :message
xtdb  |     "Cannot invoke \"java.lang.Boolean.booleanValue()\" because the return value of \"software.amazon.awssdk.services.s3.model.ListObjectsV2Response.isTruncated()\" is null",
xtdb  |     :at [xtdb.aws.S3$listAllObjects0$1 invokeSuspend "S3.kt" 202]}],
xtdb  |   :trace
xtdb  |   [[xtdb.aws.S3$listAllObjects0$1 invokeSuspend "S3.kt" 202]
xtdb  |    [kotlin.coroutines.jvm.internal.BaseContinuationImpl
xtdb  |     resumeWith
xtdb  |     "ContinuationImpl.kt"
xtdb  |     33]
xtdb  |    [kotlin.sequences.SequenceBuilderIterator
xtdb  |     hasNext
xtdb  |     "SequenceBuilder.kt"
xtdb  |     129]
xtdb  |    [clojure.lang.RT chunkIteratorSeq "RT.java" 537]
xtdb  |    [clojure.lang.RT seqFrom "RT.java" 567]
xtdb  |    [clojure.lang.RT seq "RT.java" 557]
xtdb  |    [clojure.core$seq__5486 invokeStatic "core.clj" 139]
xtdb  |    [clojure.core$seq__5486 invoke "core.clj" 139]
xtdb  |    [xtdb.metadata$load_blocks_metadata invokeStatic "metadata.clj" 414]
xtdb  |    [xtdb.metadata$load_blocks_metadata invoke "metadata.clj" 412]
xtdb  |    [xtdb.metadata$eval19345$fn__19347 invoke "metadata.clj" 433]
xtdb  |    [clojure.lang.MultiFn invoke "MultiFn.java" 234]
xtdb  |    [integrant.core$try_build_action invokeStatic "core.cljc" 419]
xtdb  |    [integrant.core$try_build_action invoke "core.cljc" 418]
xtdb  |    [integrant.core$build_key invokeStatic "core.cljc" 427]
xtdb  |    [integrant.core$build_key invoke "core.cljc" 423]
xtdb  |    [clojure.core$partial$fn__5931 invoke "core.clj" 2656]
xtdb  |    [clojure.core.protocols$fn__8275 invokeStatic "protocols.clj" 167]
xtdb  |    [clojure.core.protocols$fn__8275 invoke "protocols.clj" 123]
xtdb  |    [clojure.core.protocols$fn__8229$G__8224__8238
xtdb  |     invoke
xtdb  |     "protocols.clj"
xtdb  |     19]
xtdb  |    [clojure.core.protocols$seq_reduce invokeStatic "protocols.clj" 31]
xtdb  |    [clojure.core.protocols$fn__8262 invokeStatic "protocols.clj" 74]
xtdb  |    [clojure.core.protocols$fn__8262 invoke "protocols.clj" 74]
xtdb  |    [clojure.core.protocols$fn__8203$G__8198__8216
xtdb  |     invoke
xtdb  |     "protocols.clj"
xtdb  |     13]
xtdb  |    [clojure.core$reduce invokeStatic "core.clj" 6965]
xtdb  |    [clojure.core$reduce invoke "core.clj" 6947]
xtdb  |    [integrant.core$build invokeStatic "core.cljc" 453]
xtdb  |    [integrant.core$build invoke "core.cljc" 430]
xtdb  |    [integrant.core$init invokeStatic "core.cljc" 675]
xtdb  |    [integrant.core$init invoke "core.cljc" 667]
xtdb  |    [integrant.core$init invokeStatic "core.cljc" 672]
xtdb  |    [integrant.core$init invoke "core.cljc" 667]
xtdb  |    [xtdb.node.impl$open_node invokeStatic "impl.clj" 272]
xtdb  |    [xtdb.node.impl$open_node invoke "impl.clj" 269]
xtdb  |    [clojure.lang.Var invoke "Var.java" 386]
xtdb  |    [xtdb.api.Xtdb$Config open "Xtdb.kt" 76]
xtdb  |    [xtdb.api.Xtdb$Companion openNode "Xtdb.kt" 103]
xtdb  |    [xtdb.api.Xtdb openNode "Xtdb.kt" -1]
xtdb  |    [xtdb.node$start_node invokeStatic "node.clj" 89]
xtdb  |    [xtdb.node$start_node invoke "node.clj" 69]
xtdb  |    [xtdb.cli$start_node_from_command_line invokeStatic "cli.clj" 106]
xtdb  |    [xtdb.cli$start_node_from_command_line invoke "cli.clj" 91]
xtdb  |    [clojure.lang.Var invoke "Var.java" 386]
xtdb  |    [xtdb.main$_main invokeStatic "main.clj" 17]
xtdb  |    [xtdb.main$_main doInvoke "main.clj" 5]
xtdb  |    [clojure.lang.RestFn applyTo "RestFn.java" 140]
xtdb  |    [clojure.lang.Var applyTo "Var.java" 707]
xtdb  |    [clojure.core$apply invokeStatic "core.clj" 667]
xtdb  |    [clojure.main$main_opt invokeStatic "main.clj" 515]
xtdb  |    [clojure.main$main_opt invoke "main.clj" 511]
xtdb  |    [clojure.main$main invokeStatic "main.clj" 665]
xtdb  |    [clojure.main$main doInvoke "main.clj" 617]
xtdb  |    [clojure.lang.RestFn applyTo "RestFn.java" 140]
xtdb  |    [clojure.lang.Var applyTo "Var.java" 707]
xtdb  |    [clojure.main main "main.java" 40]],
xtdb  |   :cause
xtdb  |   "Cannot invoke \"java.lang.Boolean.booleanValue()\" because the return value of \"software.amazon.awssdk.services.s3.model.ListObjectsV2Response.isTruncated()\" is null"}}
xtdb  |
xtdb  | Execution error (NullPointerException) at xtdb.aws.S3$listAllObjects0$1/invokeSuspend (S3.kt:202).
xtdb  | Cannot invoke "java.lang.Boolean.booleanValue()" because the return value of "software.amazon.awssdk.services.s3.model.ListObjectsV2Response.isTruncated()" is null
xtdb  |
xtdb exited with code 1

Hey @tempire thanks for reporting the issue, I can’t think what may have gone wrong here asides from some change in the AWS SDK that MinIO might not yet support :thinking:

We have reasonable coverage of the MinIO Docker dependency (but using the AWS Java SDK) already in these tests xtdb/src/test/clojure/xtdb/aws/minio_test.clj at main · xtdb/xtdb · GitHub

I’ve raised an issue for further investigation: MinIO NPE at ListObjectsV2Response.isTruncated() · Issue #4201 · xtdb/xtdb · GitHub

In the meantime, please could you confirm the MinIO version you were using here?