Conduktor CLI — automate Kafka with command line

brew install conduktor/brew/conduktor-cli

docker pull conduktor/conduktor-ctl

brew upgrade conduktor/brew/conduktor-cli

docker pull conduktor/conduktor-ctl

export CDK_BASE_URL=http://localhost:8080
export CDK_API_KEY=<your-API-key>

export CDK_BASE_URL=http://localhost:8080
export CDK_USER=admin@company.io
export CDK_PASSWORD=adminP4ss!

export CDK_GATEWAY_BASE_URL=http://localhost:8888
export CDK_GATEWAY_USER=admin
export CDK_GATEWAY_PASSWORD=conduktor

Usage:
  conduktor [flags]
  conduktor [command]

Available commands:
  apply              Upsert a resource on Conduktor
  completion         Generate the autocompletion script for the specified shell
  delete             Delete resource of a given kind and name
  get                Get resource of a given kind
  help               Help about any command
  login              Login user using username password to get a JWT token
  run                run an action
  sql
  template           Get a .yaml example for a given kind
  token              Manage Admin and Application Instance tokens
  version            Display the version of Conduktor

Flags:
  -h, --help         help for  Conduktor
      --permissive   permissive mode, allow undefined environment variables
  -v, --verbose      show more information for debugging

Use "conduktor [command] --help" for more information about a command.

Usage:
  conduktor apply [flags]

Flags:
      --dry-run            Test potential changes without the effects being applied
  -f, --file stringArray   Specify the files or folders to apply. For folders, all .yaml or .yml files within the folder will be applied, while files in subfolders will be ignored.
  -h, --help               help for apply
      --parallelism int    Run each apply in parallel, useful when applying a large number of resources. Must be less than 100. (default 1)
  -r, --recursive          Apply all .yaml or .yml files in the specified folder and its subfolders. If not set, only files in the specified folder will be applied.

Global Flags:
      --permissive   Permissive mode, allow undefined environment variables
  -v, --verbose      Show more information for debugging

conduktor template topic
---
apiVersion: v2
kind: Topic
metadata:
    name: my-topic
    cluster: my-cluster
    labels:
        conduktor.io/application: application-a
        conduktor.io/application-instance: staging
        user-labels: I am a user label
    catalogVisibility: PUBLIC
    descriptionIsEditable: true
    description: This is a topic description
    sqlStorage:
        retentionTimeInSecond: 42
spec:
    partitions: 1
    replicationFactor: 1
    configs:
        cleanup.policy: delete
        retention.ms: '86400000'

conduktor template KafkaCluster -o definition.yml
conduktor template Topic >> definition.yml   #Appending to an existing file
vim definition.yml
conduktor apply -f definition.yml

conduktor get user --help
If name not provided it will list all resource

Usage:
  conduktor get User [name] [flags]

Aliases:
  User, user

Flags:
  -h, --help   help for User

Global Flags:
  -o, --output output   Output format. One of: json|yaml|name (default yaml)
      --permissive      Permissive mode, allow undefined environment variables
  -v, --verbose         Show more information for debugging

Usage:
  conduktor run [flags]
  conduktor run [command]

Available Commands:
  generateServiceAccountToken    Generate a token for a service account on a virtual cluster
  partnerZoneGenerateCredentials generate a token for a partner zone service account
  whoami                         Return information about the credentials configured in the CLI

Flags:
  -h, --help   help for run

Global Flags:
      --permissive   Permissive mode, allow undefined environment variables
  -v, --verbose      Show more information for debugging

conduktor apply -f resources.yaml --enable-state --state-file ./my-state.json

{
  "version": "v1",
  "lastUpdated": "2024-12-05T10:30:00Z",
  "resources": [
    {
      "apiVersion": "v1",
      "kind": "Topic",
      "metadata": { "name": "my-topic", "cluster": "my-cluster" }
    }
  ]
}

├── resources/
     ├── topics.yml          # Your topics are there
     ├── permissions.yml     # Your permissions to other apps are there

apiVersion: gateway/v2
kind: Interceptor
metadata:
  name: my-encryption-interceptor
spec:
  pluginClass: io.conduktor.gateway.interceptor.EncryptPlugin
  config:
    # Use $$ to escape environment variables in Interceptor configs
    encryptionKey: $${ENCRYPTION_KEY}
    apiToken: $${API_TOKEN}

---
apiVersion: console/v2
kind: KafkaCluster
metadata:
  name: my-cluster
spec:
  displayName: "My Kafka Cluster"
  icon: "kafka"
  color: "#000000"
  bootstrapServers: "localhost:9092"
  ignoreUntrustedCertificate: false
  properties:
    sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username="${ENV_VAR_FOR_USER}" password="${ENV_VAR_FOR_PASSWORD}";
    security.protocol: SASL_SSL
    sasl.mechanism: PLAIN
  schemaRegistry:
    type: "ConfluentLike"
    url: http://localhost:8080
    security:
      type: BasicAuth
      username: ${ENV_VAR_FOR_USER}
      password: ${ENV_VAR_FOR_PASSWORD}
    ignoreUntrustedCertificate: false
  kafkaFlavor:
    type: "Confluent"
    key: "${ENV_VAR_CONFLUENT_KEY}"
    secret: "${ENV_VAR_CONFLUENT_SECRET}"
    confluentEnvironmentId: "${ENV_VAR_CONFLUENT_ENV_ID:-dev}"
    confluentClusterId: "${ENV_VAR_CONFLUENT_CLUSTER_ID:-main}"