Partner Zones

Overview

Partner Zones allow you to share Kafka topics with external partners selectively and securely. You can:

• set up dedicated zones with customized access to Kafka topics
• create a single source of truth because data isn't duplicated
• reduce operational costs, since you don't have to keep data streams synchronized

Prerequisites

Before creating a Partner Zone, you have to:

• use Conduktor Console 1.32 or later
• use Conduktor Gateway 3.6.1 or later with the following configurations:
  • GATEWAY_SECURITY_PROTOCOL set to SASL_PLAIN or SASL_SSL (DELEGATED_SASL_* modes are not supported)
  • GATEWAY_USER_POOL_SERVICE_ACCOUNT_REQUIRED set to true
• use a service account to connect to Gateway, that can access the topics you want to share
• be logged in as an admin to Console UI, or using an admin token for the CLI
• in Console, configure your Gateway cluster and fill in the Provider tab with Gateway API credentials

Current limitations

As of version 1.32, Partner Zones have the following limitations:

• Partners will only be able to connect to your zone using Local Gateway Service Accounts.
• Passwords do not expire. If you need to revoke access to your partner, you will have to delete the Partner Zone.

Create a Partner Zone

You can create a Partner Zone from the Console UI, or the Conduktor CLI.

Console UI

Use the Console UI to create a Partner Zone in just a few steps.

warning

Currently, only admin users have access to Partner Zones.

1. In Conduktor Console, go to Settings > Partner Zones and click +New zone.
2. Start with the details:
   • Add a descriptive name for the zone.
   • The Technical ID will be auto-populated as you type in the name. This is used to identify this zone in CLI/API.
   • Service account will also be auto-generated based on the name but you can edit this as required. Service accounts are used to define permissions to Kafka resources, called ACLs (Access Control Lists).
   • (Optional) Enter the URL of your partner.
   • Enter a Description to explain your reasons/requirements for sharing data.
   • (Optional) Specify contact details of the beneficiary/recipient of this Partner Zone.
   • Select Gateway to choose the one you want to use and click Continue.
3. Choose which data to share: select the Kafka topics to include in this zone. By default, any topics that are shared, will be shared with Read access for security. You can additionally allow access to Write (this will also include read). Click Continue when done.
4. Finally, enable any required transformations or policies.
5. Review the details and if you're happy with the data you're about to share, click Create.

It will take a few moments for the zone to be created.

Once completed, the Credentials will be displayed. Copy/download and share these as required.

warning

If these credentials are lost, you may have to re-create the Partner Zone.

To view and manage all the zones you have access to, go to Settings > Partner Zones. You'll see the total number of zones and topics shared, as well as a list of zones, each showing:

• name and URL
• the number of topics shared
• Gateway details
• the status:
  • Pending: means the configuration isn't deployed or refreshed yet
  • Ready: shows that the configuration is up-to-date on Gateway
  • Failed: indicates that something unexpected happened during the creation. Check that the connected Gateway is active.
• the date the zone was last updated

Click on a Partner Zone to view its details.

Conduktor CLI

You can also use the Conduktor CLI (Command Line Interface) to create Partner Zones.

1. Save this example to file, e.g. pz.yaml:

   apiVersion: v2
   kind: PartnerZone
   metadata:
     name: external-partner-zone
   spec:
     cluster: partner1
     displayName: External Partner Zone
     url: https://partner1.com
     serviceAccount: johndoe
     topics:
       - name: topic-a
         backingTopic: kafka-topic-a
         permission: WRITE
       - name: topic-b
         backingTopic: kafka-topic-a
         permission: READ
     partner:
       name: John Doe
       role: Data analyst
       email: johndoe@company.io
       phone: 07827 837 177

2. Use Conduktor CLI to apply the configuration:

   conduktor apply -f pz.yaml

3. Check the status of the Partner Zone:

   ❯ conduktor get PartnerZone
   ---
   apiVersion: v2
   kind: PartnerZone
   metadata:
       name: john-partner-zone
       id: 332b30cd-7bda-4659-b1c1-39986986f0bd
       updatedAt: "2025-01-27T12:55:05.387368Z"
       status: PENDING
   spec:
       cluster: partner1
       displayName: Johns partner zone
       url: https://partner1.com
       serviceAccount: johndoe
       topics:
           - name: topic-a
             backingTopic: kafka-topic-a
             permission: WRITE
           - name: topic-b
             backingTopic: kafka-topic-a
             permission: READ
       partner:
           name: John Doe
           role: Data analyst
           email: johndoe@company.io
           phone: 07827 837 177

   The metadata statuscan have one of these values:

   • Pending: the configuration isn't deployed or refreshed yet
   • Ready: the configuration is up to date on Gateway
   • Failed: something unexpected happened during the creation

4. To securely connect to a Partner Zone through Kafka client, we've created a service account spec.serviceAccount. For additional security, you have to also generate a password for it:

   curl --request POST \
       --url 'http://localhost:8080/public/partner-zone/v2/$PARTNER_ZONE_NAME/generate-credentials' \
       --header "Authorization: Bearer $CDK_API_KEY"

5. Once generated, retrieve the password and create a config.properties file:

   security.protocol=SASL_PLAINTEXT
   sasl.mechanism=PLAIN
   sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username=johndoe password=${SERVICE_ACCOUNT_PASSWORD};

6. Finally, use CLI to list the topics of the Partner Zone:

   ❯ kafka-topics --bootstrap-server gateway:9094 --list --command-config config.properties
   topic-a
   topic-b

Delete a Partner Zone

Console UI

To delete a Partner Zone you can:

• go to the zone list view and click the three dots on the right-hand side then select Delete or
• in the zone details view, click the trash can in the top right corner.

Deleting a Partner Zone will remove a partner's access.

A confirmation window will pop up. Enter DELETE to confirm the deletion. This can't be undone.

Conduktor CLI

Deleting a Partner Zone will remove a partner's access to it. This can't be undone.

   conduktor delete PartnerZone {name}

Troubleshoot

What does Partner Zone status mean?

This is the status of a Partner Zone:

• Pending: the configuration isn't deployed or refreshed yet
• Ready: the configuration is up-to-date on Gateway
• Failed: something unexpected happened during the deployment. Check that the connected Gateway is active.

Does Generate password invalidate the previous credentials of service account?

No, you can't invalidate the credentials. If unsure, delete the Partner Zone and re-create it.

My Partner Zone creation failed, how do I find out what the issue is?

To check status, use the API or check Gateway/Console logs.

Related resources

• Connect to clusters
• Manage service accounts
• Gateway service accounts
• Give us feedback/request a feature