Skip to main content

Release date: 2024-08-14

warning

We are aware of a critical CVE - CVE-2024-41110 - coming from a dependency of prometheus on the console-cortex image. This CVE is related to prometheus docker metric scraping, which is not used by Conduktor.

Regardless, as soon as the prometheus team fix this issue, it will be patched immediately by Conduktor.

Features ✨

Manage Connectors using the CLI

Continuing with the Infra-as-code approach, we are happy to introduce CLI support for Connectors, providing an efficient and automated way to manage your Kafka Connect resources.

---
apiVersion: kafka/v2
kind: Connector
metadata:
connectCluster: kafka-connect
name: click.my-connector
labels:
conduktor.io/auto-restart-enabled: true
conduktor.io/auto-restart-frequency: 600
spec:
config:
connector.class: 'org.apache.kafka.connect.tools.MockSourceConnector'
tasks.max: '1'
topic: click.pageviews

Self-service support for Connectors

Application Teams can now manage their Connectors with Self-service.
From now on, you can grant ownership to connectors on Self-service Application Instance.

---
apiVersion: self-service/v1
kind: ApplicationInstance
metadata:
application: "clickstream-app"
name: "clickstream-dev"
spec:
cluster: "shadow-it"
serviceAccount: "sa-clicko"
resources:
- type: CONNECTOR
connectCluster: shadow-connect
patternType: PREFIXED
name: "click."

Enhanced UI & Graphs for Kafka Connect

We have revisited the Kafka Connect UI in multiple ways to improve your experience:

  • Connect Cluster selection screen with a preview of Connector status
  • New graphs demonstrating the state of your Connector over time

Kafka Connect Graphs

Support for High Availability (HA) Console

Multiple Console instances can now be deployed in parallel to achieve high availability.

This applies to the deployment of conduktor-console, while conduktor-console-cortex is currently limited to a single instance. The design ensures minimal impact on the cluster by assigning only one instance to handle the indexing of Kafka data used for performance monitoring.

Quality of Life improvements

  • The checkbox to skip TLS verification is now always visible
  • The YAML for Topic object now allows number in spec.configs. Previously it was mandatory to quote all numbers.
  • Self-service Topic Policies are now visible in the UI

Fixes 🔨

  • Topic Policies from Self-service are now properly enforced from the UI, as well as both the API and CLI
  • Fix Kafka Connect Cluster list showing invalid number of running tasks

Deprecation Warning: Upcoming migration from Tags to Labels 💣

With the introduction of the Self-service resource manifests, we brought customers a means to annotate all their resources with labels. Labels are more structured than the existing Conduktor tags, thereby allowing for more precise filtering capabilities, as can be seen in the Topic Catalog.

In an upcoming release, we'll perform an automatic migration from Tags to Labels.

Tags written with the naming convention <key>/<value> will automatically be added as similar labels:

  • <key>: <value>

If there is a conflict such as; a topic containing tags with the same key, that already has the target label, or is not written with this naming convention, then they will be created as follows:

tag-<value>: true

Here's an example of how tags will be migrated into labels:

# Tags:
- format/avro
- project/supplychain
- team/delivery
- color/blue
- color/red
- wikipedia
- non-prod

# Result
labels:
format: avro
project: supplychain
team: delivery
tag-color/blue: true # Because conflict on "color"
tag-color/red: true # Because conflict on "color"
tag-wikipedia: true
tag-non-prod: true

⚠️ Conduktor can help you rename tags through Customer Support
Between now and the migration, we can help you rename your tags for a smooth transition to labels.

Contact us as soon as possible if you would like support.