Skip to main content
Quick navigation

Managing Clusters

Overview​

tip

Use our interactive guide to learn how to connect your Kafka cluster, Schema Registry and Kafka Connect

The Admin section of Conduktor Platform enables you to add, update, and delete Kafka cluster configurations. By default, only users with Admin permissions will be able to view and perform actions relating to Cluster management. You will not be able to add clusters beyond your current organization limit; please use the chat box below or contact us to discuss an increase to this limit.

Cluster admin

To create a new cluster configuration, click on the "Create cluster" button in the top right-hand corner.

To edit an existing cluster configuration, select it from the list. You will then be able to adjust the name & color, technical ID, bootstrap servers, and additional properties. You can also enable Schema Registry and Kafka Connect for the cluster.

Connect to a secure Kafka cluster​

Conduktor leverages the default Apache Kafka Java Clients, and therefore we use the same configuration properties.

When the Conduktor Admin application needs to connect to a secure Kafka cluster, you must specify the values from your config.properties file.

For example:

security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username='<username>' password='<password>';
info

If your configuration references keystore or truststore path, check the Client Certificate Authentication documentation.

Connect to a Confluent cluster​

Via your Confluent cluster dashboard, select the Clients tab within Data integration.

Select Java as the language.

Create the Kafka cluster API key. You also have the option to create the Schema Registry API Key if you are using Schema Registry.

Copy the configuration to your clipboard.

From within the cluster configuration of the Admin application, Paste your configuration as additional properties.

You can test the connection and if successful, you will see a green CONNECTED label.

Click Create Configuration to save your cluster.

Connect to an Aiven cluster​

When connecting to an Aiven cluster, you have two options.

The first, is to add SSL authentication.

For this, you must specify the Access Key, Access Certificate.

You also need to add the Bootstrap server. This is labelled Service URI within the Aiven console. Since it's ssl, you need to specify a CA Certificate.

The second, is by using SASL.

In Aiven, change your authentication method by SASL.

Retrieve Service URI, User and Password.

You can add this information in Conduktor Admin.

Connect to a MSK cluster​

For connecting to MSK, you first need to create an IAM user.

After that, you have to give it permissions to connect to your MSK cluster.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["kafka:*", "kafka-cluster:*"],
      "Resource": "*"
    }
  ]
}

Finally, generate an access key.

You can fill the cluster form with all this information.