Quick navigation

Console Resources

ConsoleGroup

API Keys: Admin API Key
Managed with: API CLI Terraform Console UI

Creates a Group with members and permissions in Console

CLI
Terraform

---
apiVersion: iam/v2
kind: Group
metadata:
  name: developers-a
spec:
  displayName: "Developers Team A"
  description: "Members of the Team A - Developers"
  externalGroups: 
    - "LDAP-GRP-A-DEV"
  members:
    - member1@company.org
    - member2@company.org
  permissions:
    - resourceType: TOPIC
      cluster: shadow-it
      patternType: PREFIXED
      name: toto-
      permissions:
        - topicViewConfig
        - topicConsume
        - topicProduce

resource "conduktor_group_v2" "developers-a" {
  name = "developers-a"
  spec {
    display_name = "Developers Team A"
    description  = "Members of the Team A - Developers"
    externalGroups = [ "LDAP-GRP-A-DEV" ]
    members      = [ "member1@company.org", "member1@company.org" ]
    permissions  = [
     {
        resource_type = "TOPIC"
        cluster       = "shadow-it"
        patternType   = "PREFIXED"
        name          = "toto-"
        permissions   = ["topicViewConfig", "datamaskingView", "auditLogView"]
      }
    ]
  }
}

Groups checks:

spec.description is optional
spec.externalGroups is a list of LDAP or OIDC groups to sync with this Console Group
- Members added this way will not appear in spec.members but spec.membersFromExternalGroups instead
spec.membersFromExternalGroups is a read-only list of members added through spec.externalGroups
spec.members must be email addresses of members you wish to add to this group
spec.permissions are valid permissions as defined in Permissions

Side effect in Console & Kafka:

Console
- Members of the Group are given the associated permissions in the UI over the resources
- Members of the LDAP or OIDC groups will be automatically added or removed upon login
Kafka
- No side effect

ConsoleUser

API Keys: Admin API Key
Managed with: API CLI Terraform Console UI

Sets a User with permissions in Console

CLI
Terraform

---
apiVersion: iam/v2
kind: User
metadata:
  name: john.doe@company.org
spec:
  firstName: "John"
  lastName: "Doe"
  permissions:
    - resourceType: TOPIC
      cluster: shadow-it
      patternType: PREFIXED
      name: toto-
      permissions:
        - topicViewConfig
        - topicConsume
        - topicProduce

resource "conduktor_group_v2" "john.doe@company.org" {
  name = "john.doe@company.org"
  spec {
    firstname = "John"
    lastname  = "Doe"
    permissions  = [
     {
        resource_type = "TOPIC"
        cluster       = "shadow-it"
        patternType   = "PREFIXED"
        name          = "toto-"
        permissions   = ["topicViewConfig", "datamaskingView", "auditLogView"]
      }
    ]
  }
}

Users checks:

spec.permissions are valid permissions as defined in Permissions

Side effect in Console & Kafka:

Console
- User is given the associated permissions in the UI over the resources
Kafka
- No side effect

KafkaCluster

Creates a Kafka Cluster Definition in Console.

API Keys: Admin API Key
Managed with: API CLI Console UI

---
apiVersion: console/v2
kind: KafkaCluster
metadata:
  name: shadow-julien
spec:
  displayName: "Julien's cloud - Kafka"
  icon: "kafka"
  color: "#000000"
  bootstrapServers: "localhost:9092"
  ignoreUntrustedCertificate: false
  properties:
    sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin-secret";
    security.protocol: SASL_SSL
    sasl.mechanism: PLAIN
  schemaRegistry:
    type: "ConfluentLike"
    url: http://localhost:8080
    security:
      type: BasicAuth
      username: some_user
      password: some_password
    ignoreUntrustedCertificate: false
  kafkaFlavor:
    type: "Confluent"
    key: "string"
    secret: "string"
    confluentEnvironmentId: "string"
    confluentClusterId: "string"

info

metadata.name, spec.displayName, spec.icon and spec.color work together to build the visual identity of the KafkaCluster throughout Console. Cluster identity

KafkaCluster checks:

spec.icon (optional, default kafka) is a valid entry from our Icon Sets
spec.color (optional, default #000000) is a HEX color for spec.icon
spec.ignoreUntrustedCertificate (optional, default false) must be one of [true, false]
spec.schemaRegistry.type (optional) must be one of [ConfluentLike, Glue]
- See Schema Registry Properties for the detailed list of options
spec.kafkaFlavor.type (optional) must be one of [Confluent, Aiven, Gateway]
- See Kafka Provider Properties for the detailed list of options

Important

Conduktor CLI does not verify that your Kafka configuration (spec.bootstrapServers, spec.properties, ...) is valid.
You need to check that in Console directly.

Schema Registry

This section lets you associate a Schema Registry to your KafkaCluster

Confluent or Confluent-like Registry

spec:
  schemaRegistry:
    type: "ConfluentLike"
    urls: http://localhost:8080
    ignoreUntrustedCertificate: false
    security:
      type: BasicAuth
      username: some_user
      password: some_password

Confluent Schema Registry checks:

spec.schemaRegistry.urls must be a single URL of a Kafka Connect cluster
- Multiple URLs are not supported for now. Coming soon
spec.schemaRegistry.ignoreUntrustedCertificate (optional, default false) must be one of [true, false]
spec.schemaRegistry.properties (optional) is Java Properties formatted key values to further configure the SchemaRegistry
spec.security.type (optional) must be one of [BasicAuth, BearerToken, SSLAuth]
- See HTTP Security Properties for the detailed list of options

AWS Glue Registry

spec:
  schemaRegistry:
    type: "Glue"
    region: eu-west-1
    registryName: default
    security:
      type: Credentials
      accessKeyId: accessKey
      secretKey: secretKey

AWS Glue Registry checks:

spec.schemaRegistry.region must be a valid AWS region
spec.schemaRegistry.registryName must be a valid AWS Glue Registry in this region
spec.schemaRegistry.security.type must be one of [Credentials, FromContext, FromRole]

Credentials
Use AWS API Key/Secret to connect to the Glue Registry

    security:
      type: Credentials
      accessKeyId: AKIAIOSFODNN7EXAMPLE
      secretKey: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

FromContext

    security:
      type: FromContext
      profile: default

FromRole

    security:
      type: FromRole
      role: arn:aws:iam::123456789012:role/example-role

Kafka Provider

This section lets you configure the Kafka Provider for this KafkaCluster.

Confluent Cloud
Provide your Confluent Cloud details to get additional features in Console:

Confluent Cloud Service Accounts support
Confluent Cloud API Keys support

spec:
  kafkaFlavor:
    type: "Confluent"
    key: "yourApiKey123456"
    secret: "yourApiSecret123456"
    confluentEnvironmentId: "env-12345"
    confluentClusterId: "lkc-67890"

Aiven
Provide your Aiven Cloud details to get additional features in Console:

Aiven Service Accounts support
Aiven ACLs support

spec:
  kafkaFlavor:
    type: "Aiven"
    apiToken: "a1b2c3d4e5f6g7h8i9j0"
    project: "my-kafka-project"
    serviceName: "my-kafka-service"

Gateway
Provide your Gateway details to get additional features in Console:

Interceptors support

spec:
  kafkaFlavor:
    type: "Gateway"
    url: "http://gateway:8088"
    user: "admin"
    password: "admin"
    virtualCluster: vc1

Icon Sets


`cloud-bolt`	`cloud-rainbow`	`cloud`	`snowflake`


`poo-storm`	`poop`	`bolt`	`umbrella`


`tennis-ball`	`rugby-ball`	`traffic-cone`	`faucet`


`basket-shopping`	`box`	`scale-balanced`	`sunglasses`


`sword`	`axe-battle`	`vial`	`feather-pointed`


`bomb`	`flag`	`heart`	`key`


`fire-extinguisher`	`fire-flame-curved`	`alien`	`helmet-battle`


`ghost`	`robot`	`dog`	`elephant`


`bird`	`crab`	`cat-space`	`planet-ringed`


`meteor`	`moon`	`space-station`	`rocket-launch`


`paper-plane`	`car-side`	`building-columns`	`castle`


`acorn`	`burger-lettuce`	`croissant`	`mug`


`cactus`	`clover`	`camera-cctv`	`calendar`


`alarm-clock`	`compass`	`gamepad-modern`	`server`


`shield-blank`	`computer-classic`	`dharmachakra`	`kafka`

KafkaConnectCluster

Creates a Kafka Connect Cluster Definition in Console.

API Keys: Admin API Key
Managed with: API CLI Console UI

---
apiVersion: console/v2
kind: KafkaConnectCluster
metadata:
  cluster: shadow-julien
  name: connect-1
spec:
  displayName: "Connect 1"
  urls: "http://localhost:8083"
  ignoreUntrustedCertificate: false
  headers:
    X-PROJECT-HEADER: value
    AnotherHeader: test
  security:
    type: "BasicAuth"
    username: "toto"
    password: "my-secret"

KafkaConnectCluster checks:

metadata.cluster must be a valid KafkaCluster name
spec.urls must be a single URL of a Kafka Connect cluster
- Multiple URLs are not supported for now. Coming soon
spec.ignoreUntrustedCertificate (optional, default false) must be one of [true, false]
spec.headers (optional) must be key-value pairs of HTTP Headers
spec.security.type (optional) must be one of [BasicAuth, BearerToken, SSLAuth]
- See HTTP Security Properties for the detailed list of options

KsqlDBCluster

API Keys: Admin API Key
Managed with: API CLI Console UI

Creates a ksqlDB Cluster Definition in Console.

---
apiVersion: console/v2
kind: KsqlDBCluster
metadata:
  cluster: julien-cloud
  name: ksql-1
spec:
  displayName: "KSQL 1"
  url: "http://localhost:8088"
  ignoreUntrustedCertificate: false
  security:
    type: "BasicAuth"
    username: "toto"
    password: "my-secret"

KafkaConnectCluster checks:

metadata.cluster must be a valid KafkaCluster name
spec.url must be a single URL of a KsqlDB cluster
spec.ignoreUntrustedCertificate (optional, default false) must be one of [true, false]
spec.headers (optional) must be key-value pairs of HTTP Headers
spec.security.type (optional) must be one of [BasicAuth, BearerToken, SSLAuth]
- See HTTP Security Properties for the detailed list of options

Alert

API Keys: Admin API Key
Managed with: API CLI Console UI

Creates an Alert in Console.

---
apiVersion: console/v2
kind: Alert
metadata:
  cluster: local-julien
  name: my-alert
spec:
  type: TopicAlert
  topicName: wikipedia-parsed-DLQ
  metric: MessageCount
  operator: GreaterThan
  threshold: 0
  disable: false

Alert checks:

metadata.cluster must be a valid KafkaCluster name
spec.type must be one of [BrokerAlert,TopicAlert,KafkaConnectAlert]
- Check the section below for the additional mandatory fields needed for each spec.type
spec.metric is depending on the spec.type
- Check section below
spec.operator must be one of [GreaterThan, GreaterThanOrEqual, LessThan, LessThanOrEqual, NotEqual]
spec.threshold must be a number
spec.disable (optional, default false) must be one of [true, false]

When spec.type is BrokerAlert

spec.metric must be one of [MessageIn, MessageOut, MessageSize, OfflinePartitionCount, PartitionCount, UnderMinIsrPartitionCount, UnderReplicatedPartitionCount]

When spec.type is TopicAlert

spec.metric must be one of [MessageCount, MessageIn, MessageOut, MessageSize]
spec.topicName must be a Kafka Topic

When spec.type is KafkaConnectAlert

spec.metric must be FailedTaskCount
spec.connectName must be a valid KafkaConnect Cluster associated to this meta.cluster Kafka Cluster
spec.connectorName must be a Kafka Connect Connector

DataMaskingPolicy

Not implemented yet

This concept will be available in a future version

HTTP Security Properties

HTTP Security Properties are used in KafkaCluster (Schema Registry), KafkaConnect, KsqlDBCluster

Basic Authentication

  security:
    type: "BasicAuth"
    username: "toto"
    password: "my-secret"

Bearer Token

  security:
    type: "BearerToken"
    token: "toto"

mTLS / Client Certificate

  security:
    type: "SSLAuth"
    key: |
      -----BEGIN PRIVATE KEY-----
      MIIOXzCCDUegAwIBAgIRAPRytMVYJNUgCbhnA+eYumgwDQYJKoZIhvcNAQELBQAw
      ...
      IFyCs+xkcgvHFtBjjel4pnIET0agtbGJbGDEQBNxX+i4MDA=
      -----END PRIVATE KEY-----
    certificateChain: |
      -----BEGIN CERTIFICATE-----
      MIIOXzCCDUegAwIBAgIRAPRytMVYJNUgCbhnA+eYumgwDQYJKoZIhvcNAQELBQAw
      RjELMAkGA1UEBhMCVVMxIjAgBgNVBAoTGUdvb2dsZSBUcnVzdCBTZXJ2aWNlcyBM
      ...
      8/s+YDKveNdoeQoAmGQpUmxhvJ9rbNYj+4jiaujkfxT/6WtFN8N95r+k3W/1K4hs
      IFyCs+xkcgvHFtBjjel4pnIET0agtbGJbGDEQBNxX+i4MDA=
      -----END CERTIFICATE-----

Permissions

Permissions are used in Groups and Users and lets you configure all the access to any Kafka resource or Console feature.

A permission applies to a certain resourceType, which affect the necessary fields as detailed below.

Topic Permissions
Subject Permissions
ConsumerGroup Permissions
Cluster Permissions
KafkaConnect Permissions
KsqlDB Permissions
Platform Permissions

Topic Permissions

# Grants Consume, Produce and View Config to all topics toto-* on shadow-it cluster
- resourceType: TOPIC
  cluster: shadow-it
  patternType: PREFIXED
  name: toto-
  permissions:
    - topicViewConfig
    - topicConsume
    - topicProduce

resourceType: TOPIC
cluster is a valid Kafka cluster
patternType is either PREFIXED or LITERAL
name is the name of the topic or topic prefix to apply the permissions to
permissions is a list of valid topic permissions (See Table)

Available Topic Permissions	Description
`topicConsume`	Permission to consume messages from the topic.
`topicProduce`	Permission to produce (write) messages to the topic.
`topicViewConfig`	Permission to view the topic configuration.
`topicEditConfig`	Permission to edit the topic configuration.
`topicCreate`	Permission to create a new topic.
`topicDelete`	Permission to delete the topic.
`topicAddPartition`	Permission to add partitions to the topic.
`topicEmpty`	Permission to empty (delete all messages from) the topic.

Subject Permissions

# Grants View and Edit Compatibility to all subjects starting with sub-* on shadow-it cluster
- resourceType: SUBJECT
  cluster: shadow-it
  patternType: PREFIXED
  name: sub-
  permissions:
    - subjectView
    - subjectEditCompatibility

resourceType: SUBJECT
cluster is a valid Kafka cluster
patternType is either PREFIXED or LITERAL
name is the name of the subject or subject prefix to apply the permissions to
permissions is a list of valid subject permissions (See Table)

Available Subject Permissions	Description
`subjectCreateUpdate`	Permission to create or update the subject.
`subjectDelete`	Permission to delete the subject.
`subjectEditCompatibility`	Permission to edit the subject compatibility settings.
`subjectView`	Permission to view the subject details.

ConsumerGroup Permissions

# Grants View and Reset on all consumer groups starting with group-* on shadow-it cluster
- resourceType: CONSUMER_GROUP
  cluster: shadow-it
  patternType: PREFIXED
  name: group-
  permissions:
    - consumerGroupView
    - consumerGroupReset

resourceType: CONSUMER_GROUP
cluster is a valid Kafka cluster
patternType is either PREFIXED or LITERAL
name is the name of the consumer group or consumer group prefix to apply the permissions to
permissions is a list of valid consumer group permissions (See Table)

Available ConsumerGroup Permissions	Description
`consumerGroupCreate`	Permission to create a new consumer group.
`consumerGroupReset`	Permission to reset the consumer group.
`consumerGroupDelete`	Permission to delete the consumer group.
`consumerGroupView`	Permission to view the consumer group details.

Cluster Permissions

# Grants View Broker, Edit Schema Registry Compatibility, Edit Broker, View ACL, and Manage ACL on shadow-it cluster
- resourceType: CLUSTER
  name: shadow-it
  permissions:
    - clusterViewBroker
    - clusterEditSRCompatibility
    - clusterEditBroker
    - clusterViewACL
    - clusterManageACL

resourceType: CLUSTER
name is the name of the cluster to apply the permissions to
- Use * for all clusters
permissions is a list of valid cluster permissions (See Table)

Available Cluster Permissions	Description
`clusterViewBroker`	Permission to view broker details.
`clusterEditSRCompatibility`	Permission to edit Schema Registry compatibility settings.
`clusterEditBroker`	Permission to edit broker configuration.
`clusterViewACL`	Permission to view Access Control Lists (ACLs) for the cluster.
`clusterManageACL`	Permission to manage Access Control Lists (ACLs) for the cluster.

KafkaConnect Permissions

# Grants Create and Delete on all connectors starting with connector-* on shadow-it cluster and kafka-connect-cluster
- resourceType: KAFKA_CONNECT
  cluster: shadow-it
  kafkaConnect: kafka-connect-cluster
  patternType: PREFIXED
  name: connector-
  permissions:
    - kafkaConnectorCreate
    - kafkaConnectorDelete

resourceType: KAFKA_CONNECT
cluster is a valid Kafka cluster
kafkaConnect is a valid Kafka Connect cluster
patternType is either PREFIXED or LITERAL
name is the name of the connector or connector prefix to apply the permissions to
permissions is a list of valid Kafka Connect permissions (See Table)

Available KafkaConnect Permissions	Description
`kafkaConnectorViewConfig`	Permission to view the Kafka Connect configuration.
`kafkaConnectorStatus`	Permission to view the status of Kafka Connect connectors.
`kafkaConnectorEditConfig`	Permission to edit the Kafka Connect configuration.
`kafkaConnectorDelete`	Permission to delete connectors.
`kafkaConnectorCreate`	Permission to create new connectors.
`kafkaConnectPauseResume`	Permission to pause and resume connectors.
`kafkaConnectRestart`	Permission to restart connectors.

KsqlDB Permissions

# Grants all permissions on KsqlDB cluster ksql-cluster
- resourceType: KSQLDB
  cluster: shadow-it
  ksqlDB: ksql-cluster
  permissions:
    - ksqldbAccess

resourceType: KSQLDB
cluster is a valid Kafka cluster
ksqlDB is a valid Kafka Connect cluster
permissions is a list of valid KsqlDB permissions (See Table)

Available KafkaConnect Permissions	Description
`ksqldbAccess`	Grants all permissions on the KsqlDB Cluster.

Platform Permissions

# Grants Platform permissions
- resourceType: PLATFORM
  permissions:
    - userView
    - datamaskingView

resourceType: PLATFORM
permissions is a list of valid Platform permissions (See Table)

Available KafkaConnect Permissions	Description
`clusterConnectionsManage`	Permission to add / edit / remove Kafka clusters on Console
`certificateManage`	Permission to add / edit / remove TLS Certificates on Console
`userManage`	Permission to manage Console users, groups & permissions
`userView`	Permission to view Console users, groups & permissions
`datamaskingManage`	Permission to manage Data policies (masking rules)
`datamaskingView`	Permission to view Data policies
`notificationChannelManage`	Permission to manage Integration channels
`notificationChannelView`	Permission to view Integration channels
`auditLogView`	Permission to browse audit log

ConsoleGroup​

ConsoleUser​

KafkaCluster​

Schema Registry​

Confluent or Confluent-like Registry​

AWS Glue Registry​

Kafka Provider​

Icon Sets​

KafkaConnectCluster​

KsqlDBCluster​

Alert​

DataMaskingPolicy​

HTTP Security Properties​

Basic Authentication​

Bearer Token​

mTLS / Client Certificate​

Permissions​

Topic Permissions​

Subject Permissions​

ConsumerGroup Permissions​

Cluster Permissions​

KafkaConnect Permissions​

KsqlDB Permissions​

Platform Permissions​

ConsoleGroup

ConsoleUser

KafkaCluster

Schema Registry

Confluent or Confluent-like Registry

AWS Glue Registry

Kafka Provider

Icon Sets

KafkaConnectCluster

KsqlDBCluster

Alert

DataMaskingPolicy

HTTP Security Properties

Basic Authentication

Bearer Token

mTLS / Client Certificate

Permissions

Topic Permissions

Subject Permissions

ConsumerGroup Permissions

Cluster Permissions

KafkaConnect Permissions

KsqlDB Permissions

Platform Permissions