Skip to main content

​
Overview

To add, update, and delete Kafka cluster configurations, go to Settings > Clusters. By default, only users belonging to the Admin group or having the Can manage Cluster configurations permission will be able to view and manage the clusters. Cluster admin To add a cluster: Click Add cluster in the top right corner. To edit a cluster: Select it from the list to modify the name, icon, bootstrap servers, and additional properties. You can also enable Schema Registry, Kafka Connect, ksqlDB, and Kafka providers.

​
Connect to a Conduktor Gateway cluster

Connect Conduktor Console to a Conduktor Gateway cluster the same way you would connect to any Kafka cluster by providing the Gateway bootstrap servers and security configurations. Console will simply act as a Kafka client connecting to Gateway. The security protocol must match the GATEWAY_SECURITY_PROTOCOL defined in your Gateway configuration. To enable Gateway integration and access interceptors, Chargeback, or Partner Zones, select the Gateway provider and enter your Gateway API key and secret. Gateway provider in Console

​
Connect to a secure Kafka cluster

Console uses the Apache Kafka Java Client configuration properties . To connect to a secure Kafka cluster, specify the values from your config.properties file: 
security.protocol=SASL_SSL
sasl.mechanism=SCRAM-SHA-512
sasl.jaas.config=org.apache.kafka.common.security.scram.ScramLoginModule required username='<username>' password='<password>';

​
Connect to a Confluent Cloud cluster

  1. In your Confluent cluster dashboard, go to Data integration > Clients
  2. Select Java as the language
Confluent client
  1. Create a Kafka cluster API key. Optionally, create a Schema Registry API Key if using Schema Registry
Confluent snippet
  1. Copy the configuration to your clipboard.
  2. In Console, add the bootstrap servers and paste the configuration as advanced properties.
  3. Test the connection. A green Connected label appears on success.
Confluent configuration in Console
  1. Click Create Configuration to save
To manage Confluent service accounts, set up the Confluent provider in the cluster configuration.

​
Connect to an Aiven cluster

Choose one of the following authentication methods:

​
SSL authentication

Provide the Access Key, Access Certificate, and CA Certificate from your Aiven console. The Service URI is your bootstrap server. Aiven certificates In Console:
  1. Enter the bootstrap server
  2. Upload the CA certificate
  3. Select SSL as the authentication method
  4. Paste the Access Key and Access Certificate
Console config with SSL To manage Aiven service accounts, set up the Aiven provider in the cluster configuration.

​
SASL_SSL authentication

Provide the Service URI (bootstrap server), User, and Password from your Aiven console. Aiven SASL The configuration in Console: Console config with SASL_SSL To manage Aiven service accounts, set up the Aiven provider in the cluster configuration.

​
Connect to an AWS MSK cluster

  1. Create an IAM user:
MSK IAM user
  1. Attach permissions to the IAM user:
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["kafka:*", "kafka-cluster:*"],
      "Resource": "*"
    }
  ]
}
  1. Generate an access key:
MSK Access key
  1. In Console, enter the Bootstrap server and select AWS IAM as the authentication method
  2. Choose one option:
    • Inherit credentials from environment variables, or
    • Enter the Access Key and Secret Key
Console config with IAM

​
Connect to a Cloudera cluster

These instructions use SASL_SSL protocol with PLAIN mechanism.
  1. Create a workload user with ownership of the Data Hub cluster. Note the username and password:
cloudera-user-management
  1. Download certificates from Cloudera:
getting_certs_from_cloudera
  1. Convert CRT certificates to JKS format using Java keytool:
keytool -import -keystore zeke-test2-cdp-env.jks -alias zeke-test2-cdp-env -file zeke-test2-cdp-env.crt
  1. In Cloudera, open firewalls for Kafka brokers and Schema Registry
  2. In Console, go to Clusters, select the Cloudera cluster, and upload the certificates
  3. Configure the cluster using the workload username and password from step 1:
adding cloudera to console Automated setup example

​
Connect to a Google Cloud cluster

Connect to Google Cloud Managed Service for Apache Kafka using SASL_SSL protocol with PLAIN mechanism.

​
Use a service account

View Google Cloud documentation

​
Use an access token

  1. Get an access token:
gcloud auth login --no-launch-browser
gcloud auth print-access-token
  1. Use the token with these parameters:
security.protocol=SASL_SSL
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
username="PRINCIPAL_EMAIL_ADDRESS" \
password="ACCESS_TOKEN_VALUE";
Managed Service for Apache Kafka verifies that:
  • The access token is valid and not expired
  • The username matches the principal email associated with the access token
  • The principal has the managedkafka.clusters.connect permission (included in roles/managedkafka.client)