We are trying to evaluate XTDB in a research context. We use self-hosted Kubernetes clusters for all our deployments.
I can only find setup guides and files relating to an AWS setup. Am I missing critical docs or is setup documentation for these types of environments still to do?
Would it be possible to deploy XTDB and configure it to use our self-hosted MinIO S3 compatible storage and Apache Kafka?
Hey @MisterTimn we are currently working with a small group of Design Partners to support deployments into public cloud environments as the priority, hence the focus on AWS and more recently Google Cloud (you can see the ongoing work here which includes GKS, Google’s managed Kubernetes service).
However, we do also want to support self-hosting XT2 outside of public clouds, and therefore a combination of MinIO S3 compatible storage + Apache Kafka should be viable and we will document our recommendations for achieving that in due course.
Feel free to get in touch if you would like to understand more about our plans or help shape them: jdt@juxt.pro - and please also consider applying to become a Design Partner
We are trying to evaluate XTDB in a research context.
May I ask which other systems you are evaluating against?
This is our first experience with immutable databases. No concrete plans or use cases yet, but it is interesting to compare XTDB with a proof-of-concept we’ve made using Clickhouse to store and query state revisions.
Am I correct in assuming that the new snapshot has paved the way for a self hosted deployment of xtdb?
Main change here is a change to the XT2 infrastructural reqmts - previously, on cloud deployments, we required that the object store be able to notify the nodes when new files were added; we now take responsibility for this ourselves by adding an extra topic on the existing log, so there’s no longer any need to set up extra pub/sub infrastructure.
Seeing as the only requirements now are s3 compatible storage (e.g. MinIO) and Kafka?