Setup guide for self hosted / plain Kubernetes clusters?

We are trying to evaluate XTDB in a research context. We use self-hosted Kubernetes clusters for all our deployments.

I can only find setup guides and files relating to an AWS setup. Am I missing critical docs or is setup documentation for these types of environments still to do?

Would it be possible to deploy XTDB and configure it to use our self-hosted MinIO S3 compatible storage and Apache Kafka?

1 Like

Hey @MisterTimn we are currently working with a small group of Design Partners to support deployments into public cloud environments as the priority, hence the focus on AWS and more recently Google Cloud (you can see the ongoing work here which includes GKS, Google’s managed Kubernetes service).

However, we do also want to support self-hosting XT2 outside of public clouds, and therefore a combination of MinIO S3 compatible storage + Apache Kafka should be viable and we will document our recommendations for achieving that in due course.

Feel free to get in touch if you would like to understand more about our plans or help shape them: jdt@juxt.pro - and please also consider applying to become a Design Partner :slightly_smiling_face:

We are trying to evaluate XTDB in a research context.

May I ask which other systems you are evaluating against?

2 Likes

This is our first experience with immutable databases. No concrete plans or use cases yet, but it is interesting to compare XTDB with a proof-of-concept we’ve made using Clickhouse to store and query state revisions.

1 Like

Am I correct in assuming that the new snapshot has paved the way for a self hosted deployment of xtdb?

Main change here is a change to the XT2 infrastructural reqmts - previously, on cloud deployments, we required that the object store be able to notify the nodes when new files were added; we now take responsibility for this ourselves by adding an extra topic on the existing log, so there’s no longer any need to set up extra pub/sub infrastructure.

Seeing as the only requirements now are s3 compatible storage (e.g. MinIO) and Kafka?

1 Like

It wasn’t the main objective behind that change, but yes it should make it much simpler to deploy XT2 outside of the main cloud vendors :slightly_smiling_face: