Conduktor CLI overview

Conduktor CLI (Command Line Interface) helps you to manage your Console and Gateway resources via commands or CI/CD pipelines.

Install

You have three options to install Conduktor CLI:
  • Native binary - for individual use and testing
  • Homebrew - for individual use and testing on MacOS
  • Docker - for integration in CI/CD pipelines

Native binary

Download the latest binary from GitHub on Windows, Linux or MacOS. In the Assets list, download the build that corresponds to your machine (for Apple Silicon use darwin-arm64).

Homebrew

Use the Homebrew package manager to install our CLI:
brew install conduktor/brew/conduktor-cli

Docker

Pull the latest Dockerhub version of the CLI using the following command line:
docker pull conduktor/conduktor-ctl

Configure

To use the Conduktor CLI, you need to set:
  • the URL of your Conduktor Console and/or Gateway API
  • the authentication mechanism: either by using an API key, or user credentials

Authenticate to Console using API keys

These are the necessary environment variables for this authentication method::
export CDK_BASE_URL=http://localhost:8080
export CDK_API_KEY=<your-API-key>
There are two types of API keys: admin and Self-service application.
Admin API keys grant the maximum permissions in Console.
Generate an admin key named 'my-admin-key'
conduktor token create admin my-admin-key

Authenticate to Console using user credentials

You can simply set the username and password of the local or LDAP user and use the CLI, or create an associated short-lived API key. This type of API key will have the permissions of the user who created it. This kind of authentication will be valid for the same duration as the configured session timeout.
export CDK_BASE_URL=http://localhost:8080
export CDK_USER=admin@company.io
export CDK_PASSWORD=adminP4ss!
Generate a short-lived API key
conduktor login

Authenticate to Console using external authentication

By default, the Conduktor CLI authenticates with Console using a Console API key, or generates one based on the given credentials. This is the console authentication mode (CDK_AUTH_MODE is console or empty). Alternatively, you can use the external authentication mode to bypass token generation and use credentials from your identity provider or the API proxy instead. For that, set the CDK_AUTH_MODE environment variable to external:
Set up external authentication with a bearer token
export CDK_AUTH_MODE="external"
export CDK_API_KEY=<bearer_token>
Set up external authentication with client credentials
export CDK_AUTH_MODE="external"
export CDK_USER=<client_id>
export CDK_PASSWORD=<client_secret>

Authenticate to Gateway using API credentials

To use the CLI to manage Gateway objects, you need to set the URL of the Gateway API and the API credentials.
export CDK_GATEWAY_BASE_URL=http://localhost:8888
export CDK_GATEWAY_USER=admin
export CDK_GATEWAY_PASSWORD=conduktor

Environment variables

This is the list of the supported environment variables:
Environment variablesDescription
Console
CDK_BASE_URLConsole base URL, e.g. http://localhost:8080
CDK_API_KEYConsole API key (admin or application key)
CDK_USERConsole local or LDAP user username
CDK_PASSWORDConsole local or LDAP user password
CDK_AUTH_MODEAuthentication mode (external, console or empty (default))
Gateway
CDK_GATEWAY_BASE_URLGateway base URL, e.g. http://localhost:8888
CDK_GATEWAY_USERGateway user login email
CDK_GATEWAY_PASSWORDGateway user login password
TLS & mTLS
CDK_CACERTRoot CA certificate to verify the Conduktor Console certificate (.pem)
CDK_INSECURESkip the TLS verification of the Conduktor Console. Defaults to false.
CDK_CERTCertificate to authenticate the CLI (.pem)
CDK_KEYKey to authenticate the CLI (.pem)

CLI commands

The CLI offers a set of commands to manage Conduktor resources. You can use it to create, update, delete and get resources.
Usage:
  conduktor [flags]
  conduktor [command]

Available commands:
  apply              Upsert a resource on Conduktor
  completion         Generate the autocompletion script for the specified shell
  delete             Delete resource of a given kind and name
  get                Get resource of a given kind
  help               Help about any command
  login              Login user using username password to get a JWT token
  run                run an action
  sql
  template           Get a .yaml example for a given kind
  token              Manage Admin and Application Instance tokens
  version            Display the version of Conduktor

Flags:
  -h, --help         help for  Conduktor
      --permissive   permissive mode, allow undefined environment variables
  -v, --verbose      show more information for debugging

Use "conduktor [command] --help" for more information about a command.

Apply

The apply command allows you to deploy any resource.
Usage:
  conduktor apply [flags]

Flags:
      --dry-run            Test potential changes without the effects being applied
  -f, --file stringArray   Specify the files or folders to apply. For folders, all .yaml or .yml files within the folder will be applied, while files in subfolders will be ignored.
  -h, --help               help for apply
      --parallelism int    Run each apply in parallel, useful when applying a large number of resources. Must be less than 100. (default 1)
  -r, --recursive          Apply all .yaml or .yml files in the specified folder and its subfolders. If not set, only files in the specified folder will be applied.

Global Flags:
      --permissive   Permissive mode, allow undefined environment variables
  -v, --verbose      Show more information for debugging
The --parallelism flag allows you to run multiple applies at the same time, in parallel. This is useful when applying a large number of resources. Be careful not to set it too high, as it may overload the Console/Gateway API.
Examples:
Apply all resources in multiple directories
conduktor apply -f ./directory1 -f ./directory2
Apply a specific file
conduktor apply -f file.yaml
Apply all resources in the current directory and its subfolders
conduktor apply -f . -r

Template

The template command allows you to get an example of a resource definition:
conduktor template topic
---
apiVersion: v2
kind: Topic
metadata:
    name: my-topic
    cluster: my-cluster
    labels:
        conduktor.io/application: application-a
        conduktor.io/application-instance: staging
        user-labels: I am a user label
    catalogVisibility: PUBLIC
    descriptionIsEditable: true
    description: This is a topic description
    sqlStorage:
        retentionTimeInSecond: 42
spec:
    partitions: 1
    replicationFactor: 1
    configs:
        cleanup.policy: delete
        retention.ms: '86400000'
You can also save the output in a file, in order to edit it before applying it:
conduktor template KafkaCluster -o definition.yml
conduktor template Topic >> definition.yml   #Appending to an existing file
vim definition.yml
conduktor apply -f definition.yml

Delete

The delete command allows you to delete a resource. It works the same way as the apply.
Resources are deleted instantly and can’t be recovered. Any data or access associated with the resource is permanently lost.
Examples:
Delete all resources in multiple directories
conduktor delete -f ./directory1 -f ./directory2
Delete a specific file
conduktor delete -f file.yaml
Delete all resources in the current directory and its subfolders
conduktor delete -f . -r

Get

The get command allows you to retrieve resources. You can specify a kind, a name, or list all the resources.
conduktor get user --help
If name not provided it will list all resource

Usage:
  conduktor get User [name] [flags]

Aliases:
  User, user

Flags:
  -h, --help   help for User

Global Flags:
  -o, --output output   Output format. One of: json|yaml|name (default yaml)
      --permissive      Permissive mode, allow undefined environment variables
  -v, --verbose         Show more information for debugging
Examples:
Get all users
conduktor get user
Get a specific Kafka connect cluster
conduktor get KafkaConnectCluster --cluster my-kafka-cluster
Get all the Console resources with a 'name' output format
conduktor get all --console -o name

Token

To run your CI/CD pipelines, you can use the token command to manage your API keys.
Create an admin API keys
conduktor token create admin my_first_admin_token
List Console admin API keys
conduktor token list admin
Create an application instance API key
conduktor token create application-instance -i=my_instance my_token_for_my_instance
List API keys of an application instance
conduktor token list application-instance -i=my_instance
Delete an API key
conduktor token delete 1233ff18-0e95-4638-b3fc-f55e20088b8d

Run

The run command allows you to run a few actions:
Usage:
  conduktor run [flags]
  conduktor run [command]

Available Commands:
  generateServiceAccountToken    Generate a token for a service account on a virtual cluster
  partnerZoneGenerateCredentials generate a token for a partner zone service account
  whoami                         Return information about the credentials configured in the CLI

Flags:
  -h, --help   help for run

Global Flags:
      --permissive   Permissive mode, allow undefined environment variables
  -v, --verbose      Show more information for debugging
Examples:
Generate a service account password for a virtual cluster
conduktor run generateServiceAccountToken --v-cluster my-vcluster --username my-service-account
Generate a Partner Zone service account password
conduktor run partnerZoneGenerateCredentials --partner-zone-name my-partner-zone
Get information about the credentials configured in the CLI
conduktor run whoami

Version

Check the current version of your CLI:
Check CLI version
conduktor version

CI/CD pipeline integration example

Conduktor CLI can be easily integrated to a CI/CD pipeline. This example presents two pipelines:
  • The first one triggers on each new PR and launches the CLI using the --dry-run flag, generating a report confirming that the resources can be successfully created or modified.
  • The second one triggers on a push to the main branch, making the changes live.
Consider the following folder structure:
├── resources/
│   ├── topics.yml          # Your topics are there
|   ├── permissions.yml     # Your permissions to other apps are there
.github/workflows/on-pr.yml

name: Check PR Validity
on:
  pull_request:
    branches: [ "main" ]
  workflow_dispatch:
jobs:
  build:
    runs-on: ubuntu-latest
    container: conduktor/conduktor-ctl
    steps:
      - uses: actions/checkout@v3
      - run: /bin/conduktor apply -f resources/ --dry-run
        env:
          CDK_BASE_URL: https://conduktor.domain.com
          CDK_API_KEY: ${{ secrets.CONDUKTOR_TOKEN }}
.github/workflows/on-push.yml
name: Execute committed changes
on:
  push:
    branches: [ "main" ]
jobs:
  build:
    runs-on: ubuntu-latest
    container: conduktor/conduktor-ctl
    steps:
      - uses: actions/checkout@v3
      - run: /bin/conduktor apply -f resources/
        env:
          CDK_BASE_URL: https://conduktor.domain.com
          CDK_API_KEY: ${{ secrets.CONDUKTOR_TOKEN }}

Environment variables for secrets

The Conduktor CLI automatically substitutes environment variables in YAML manifests using ${ENV_VAR} patterns. This keeps sensitive data like credentials out of configuration files. You can set a default value using POSIX notation ${ENV:-default}. By default, the CLI fails if an environment variable is undefined or empty, ensuring all required variables are properly configured. Use the --permissive flag to replace missing or empty variables with empty strings instead of failing. The example below shows a Conduktor cluster configuration with credentials secured using environment variables:
---
apiVersion: console/v2
kind: KafkaCluster
metadata:
  name: my-cluster
spec:
  displayName: "My Kafka Cluster"
  icon: "kafka"
  color: "#000000"
  bootstrapServers: "localhost:9092"
  ignoreUntrustedCertificate: false
  properties:
    sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username="${ENV_VAR_FOR_USER}" password="${ENV_VAR_FOR_PASSWORD}";
    security.protocol: SASL_SSL
    sasl.mechanism: PLAIN
  schemaRegistry:
    type: "ConfluentLike"
    url: http://localhost:8080
    security:
      type: BasicAuth
      username: ${ENV_VAR_FOR_USER}
      password: ${ENV_VAR_FOR_PASSWORD}
    ignoreUntrustedCertificate: false
  kafkaFlavor:
    type: "Confluent"
    key: "${ENV_VAR_CONFLUENT_KEY}"
    secret: "${ENV_VAR_CONFLUENT_SECRET}"
    confluentEnvironmentId: "${ENV_VAR_CONFLUENT_ENV_ID:-dev}"
    confluentClusterId: "${ENV_VAR_CONFLUENT_CLUSTER_ID:-main}"