Skip to main content

Tools

Let's talk about tools I believe are necessary for development around Kafka.

Schema Registry​

We'll use Confluent's schema registry in this study, as it's the most widely used. In the future, we'll do a study on Apicur.io.

We can go down two paths:

  • Use the Confluent Schema Registry offered by Bitnami.
  • Use the official one offered by Confluent itself.
    • In this case, it's necessary to install the operator that performs the installation process through custom resources.

The advantage of installing through CFK is that we're extending the Kubernetes API for future automations.

helm repo add confluentinc https://packages.confluent.io/helm
helm repo update
helm install confluent-operator confluentinc/confluent-for-kubernetes --namespace confluent --create-namespace

There's not much to change in values.yaml. Actually, we'll install some custom resources and only use a few to create the schema registry and in the future some schemas.

We can also take the opportunity to install ksqlDB which is also offered through a custom resource.

To install the schema registry, we create the following manifest.

apiVersion: platform.confluent.io/v1beta1
kind: SchemaRegistry
metadata:
  name: schemaregistry
  namespace: confluent
spec:
  replicas: 1 # Adjust for a larger cluster
  image:
    application: confluentinc/cp-schema-registry:7.5.0
    init: confluentinc/confluent-init-container:2.7.0
  dependencies:
    kafka:
      bootstrapEndpoint: kafka-cluster-kafka-bootstrap.kafka:9092
apiVersion: platform.confluent.io/v1beta1
kind: KsqlDB
metadata:
  name: ksqldb
  namespace: confluent
spec:
  replicas: 1 # Adjust for a larger cluster
  image:
    application: confluentinc/cp-ksqldb-server:7.5.0
    init: confluentinc/confluent-init-container:2.7.0
  dataVolumeCapacity: 10Gi
  dependencies:
    kafka:
      bootstrapEndpoint: kafka-cluster-kafka-bootstrap.kafka:9092
    schemaRegistry:
      url: http://schemaregistry.confluent:8081