Release date: 2024-08-14

Features ✨​

• Manage Connectors using the CLI
• Self-service support for Connectors
• Enhanced UI & Alerts for Kafka Connect
• Quality of Life improvements
• Deprecation Warning: Upcoming migration from Tags to Labels

Manage Connectors using the CLI​

Continuing with the Infra-as-code approach, we are happy to introduce CLI support for Connectors, providing an efficient and automated way to manage your Kafka Connect resources.

---
apiVersion: kafka/v2
kind: Connector
metadata:
  connectCluster: kafka-connect
  name: click.my-connector
  labels:
    conduktor.io/auto-restart-enabled: true
    conduktor.io/auto-restart-frequency: 600
spec:
  config:
    connector.class: 'org.apache.kafka.connect.tools.MockSourceConnector'
    tasks.max: '1'
    topic: click.pageviews

Self-service support for Connectors​

Application Teams can now manage their Connectors with Self-service.
From now on, you can grant ownership to connectors on Self-service Application Instance.

---
apiVersion: self-service/v1
kind: ApplicationInstance
metadata:
  application: "clickstream-app"
  name: "clickstream-dev"
spec:
  cluster: "shadow-it"
  serviceAccount: "sa-clicko"
  resources:
    - type: CONNECTOR
      connectCluster: shadow-connect
      patternType: PREFIXED
      name: "click."

Enhanced UI & Graphs for Kafka Connect​

We have revisited the Kafka Connect UI in multiple ways to improve your experience:

• Connect Cluster selection screen with a preview of Connector status
• New graphs demonstrating the state of your Connector over time

Support for High Availability (HA) Console​

Multiple Console instances can now be deployed in parallel to achieve high availability.

This applies to the deployment of conduktor-console, while conduktor-console-cortex is currently limited to a single instance. The design ensures minimal impact on the cluster by assigning only one instance to handle the indexing of Kafka data used for performance monitoring.

Quality of Life improvements​

• The checkbox to skip TLS verification is now always visible
• The YAML for Topic object now allows number in spec.configs. Previously it was mandatory to quote all numbers.
• Self-service Topic Policies are now visible in the UI

Fixes 🔨​

• Topic Policies from Self-service are now properly enforced from the UI, as well as both the API and CLI
• Fix Kafka Connect Cluster list showing invalid number of running tasks

Deprecation Warning: Upcoming migration from Tags to Labels 💣​

With the introduction of the Self-service resource manifests, we brought customers a means to annotate all their resources with labels. Labels are more structured than the existing Conduktor tags, thereby allowing for more precise filtering capabilities, as can be seen in the Topic Catalog.

In an upcoming release, we'll perform an automatic migration from Tags to Labels.

Tags written with the naming convention <key>/<value> will automatically be added as similar labels:

• <key>: <value>

If there is a conflict such as; a topic containing tags with the same key, that already has the target label, or is not written with this naming convention, then they will be created as follows:

tag-<value>: true

Here's an example of how tags will be migrated into labels:

# Tags:
- format/avro
- project/supplychain
- team/delivery
- color/blue
- color/red
- wikipedia
- non-prod

# Result
labels:
  format: avro
  project: supplychain
  team: delivery
  tag-color/blue: true # Because conflict on "color"
  tag-color/red: true # Because conflict on "color"
  tag-wikipedia: true
  tag-non-prod: true

⚠️ Conduktor can help you rename tags through Customer Support
Between now and the migration, we can help you rename your tags for a smooth transition to labels.

Contact us as soon as possible if you would like support.