Console Resources
ConsoleGroup
API Keys: Admin API Key
Managed with: API CLI Console UI
Creates a Group with members and permissions in Console
---
apiVersion: iam/v2
kind: Group
metadata:
name: developers-a
spec:
displayName: "Developers Team A"
description: "Members of the Team A - Developers"
externalGroups:
- "LDAP-GRP-A-DEV"
members:
- member1@company.org
- member2@company.org
permissions:
- resourceType: TOPIC
cluster: shadow-it
patternType: PREFIXED
name: toto-
permissions:
- topicViewConfig
- topicConsume
- topicProduce
Groups checks:
spec.description
is optionalspec.externalGroups
is a list of LDAP or OIDC groups to sync with this Console Group- Members added this way will not appear in
spec.members
butspec.membersFromExternalGroups
instead
- Members added this way will not appear in
spec.membersFromExternalGroups
is a read-only list of members added throughspec.externalGroups
spec.members
must be email addresses of members you wish to add to this groupspec.permissions
are valid permissions as defined in Permissions
Side effect in Console & Kafka:
- Console
- Members of the Group are given the associated permissions in the UI over the resources
- Members of the LDAP or OIDC groups will be automatically added or removed upon login
- Kafka
- No side effect
ConsoleUser
API Keys: Admin API Key
Managed with: API CLI Console UI
Sets a User with permissions in Console
---
apiVersion: iam/v2
kind: User
metadata:
name: john.doe@company.org
spec:
firstName: "John"
lastName: "Doe"
permissions:
- resourceType: TOPIC
cluster: shadow-it
patternType: PREFIXED
name: toto-
permissions:
- topicViewConfig
- topicConsume
- topicProduce
Users checks:
spec.permissions
are valid permissions as defined in Permissions
Side effect in Console & Kafka:
- Console
- User is given the associated permissions in the UI over the resources
- Kafka
- No side effect
KafkaCluster
Creates a Kafka Cluster Definition in Console.
API Keys: Admin API Key
Managed with: API CLI Console UI
---
apiVersion: console/v2
kind: KafkaCluster
metadata:
name: shadow-julien
spec:
displayName: "Julien's cloud - Kafka"
icon: "kafka"
color: "#000000"
bootstrapServers: "localhost:9092"
ignoreUntrustedCertificate: false
properties:
sasl.jaas.config: org.apache.kafka.common.security.plain.PlainLoginModule required username="admin" password="admin-secret";
security.protocol: SASL_SSL
sasl.mechanism: PLAIN
schemaRegistry:
type: "ConfluentLike"
url: http://localhost:8080
security:
type: BasicAuth
username: some_user
password: some_password
ignoreUntrustedCertificate: false
kafkaFlavor:
type: "Confluent"
key: "string"
secret: "string"
confluentEnvironmentId: "string"
confluentClusterId: "string"
metadata.name
, spec.displayName
, spec.icon
and spec.color
work together to build the visual identity of the KafkaCluster throughout Console.
KafkaCluster checks:
spec.icon
(optional, defaultkafka
) is a valid entry from our Icon Setsspec.color
(optional, default#000000
) is a HEX color forspec.icon
spec.ignoreUntrustedCertificate
(optional, defaultfalse
) must be one of [true
,false
]spec.schemaRegistry.type
(optional) must be one of [ConfluentLike
,Glue
]- See Schema Registry Properties for the detailed list of options
spec.kafkaFlavor.type
(optional) must be one of [Confluent
,Aiven
,Gateway
]- See Kafka Provider Properties for the detailed list of options
Conduktor CLI does not verify that your Kafka configuration (spec.bootstrapServers
, spec.properties
, ...) is valid.
You need to check that in Console directly.
Schema Registry
This section lets you associate a Schema Registry to your KafkaCluster
Confluent or Confluent-like Registry
spec:
schemaRegistry:
type: "ConfluentLike"
urls: http://localhost:8080
ignoreUntrustedCertificate: false
security:
type: BasicAuth
username: some_user
password: some_password
Confluent Schema Registry checks:
spec.schemaRegistry.urls
must be a single URL of a Kafka Connect cluster- Multiple URLs are not supported for now. Coming soon
spec.schemaRegistry.ignoreUntrustedCertificate
(optional, defaultfalse
) must be one of [true
,false
]spec.schemaRegistry.properties
(optional) is Java Properties formatted key values to further configure the SchemaRegistryspec.security.type
(optional) must be one of [BasicAuth
,BearerToken
,SSLAuth
]- See HTTP Security Properties for the detailed list of options
AWS Glue Registry
spec:
schemaRegistry:
type: "Glue"
region: eu-west-1
registryName: default
security:
type: Credentials
accessKeyId: accessKey
secretKey: secretKey
AWS Glue Registry checks:
spec.schemaRegistry.region
must be a valid AWS regionspec.schemaRegistry.registryName
must be a valid AWS Glue Registry in this regionspec.schemaRegistry.security.type
must be one of [Credentials
,FromContext
,FromRole
]
Credentials
Use AWS API Key/Secret to connect to the Glue Registry
security:
type: Credentials
accessKeyId: AKIAIOSFODNN7EXAMPLE
secretKey: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
FromContext
security:
type: FromContext
profile: default
FromRole
security:
type: FromRole
role: arn:aws:iam::123456789012:role/example-role
Kafka Provider
This section lets you configure the Kafka Provider for this KafkaCluster.
Confluent Cloud
Provide your Confluent Cloud details to get additional features in Console:
- Confluent Cloud Service Accounts support
- Confluent Cloud API Keys support
spec:
kafkaFlavor:
type: "Confluent"
key: "yourApiKey123456"
secret: "yourApiSecret123456"
confluentEnvironmentId: "env-12345"
confluentClusterId: "lkc-67890"
Aiven
Provide your Aiven Cloud details to get additional features in Console:
- Aiven Service Accounts support
- Aiven ACLs support
spec:
kafkaFlavor:
type: "Aiven"
apiToken: "a1b2c3d4e5f6g7h8i9j0"
project: "my-kafka-project"
serviceName: "my-kafka-service"
Gateway
Provide your Gateway details to get additional features in Console:
- Interceptors support
spec:
kafkaFlavor:
type: "Gateway"
url: "http://gateway:8088"
user: "admin"
password: "admin"
virtualCluster: vc1
Icon Sets
cloud-bolt | cloud-rainbow | cloud | snowflake |
poo-storm | poop | bolt | umbrella |
tennis-ball | rugby-ball | traffic-cone | faucet |
basket-shopping | box | scale-balanced | sunglasses |
sword | axe-battle | vial | feather-pointed |
bomb | flag | heart | key |
fire-extinguisher | fire-flame-curved | alien | helmet-battle |
ghost | robot | dog | elephant |
bird | crab | cat-space | planet-ringed |
meteor | moon | space-station | rocket-launch |
paper-plane | car-side | building-columns | castle |
acorn | burger-lettuce | croissant | mug |
cactus | clover | camera-cctv | calendar |
alarm-clock | compass | gamepad-modern | server |
shield-blank | computer-classic | dharmachakra | kafka |
KafkaConnectCluster
Creates a Kafka Connect Cluster Definition in Console.
API Keys: Admin API Key
Managed with: API CLI Console UI
---
apiVersion: console/v2
kind: KafkaConnectCluster
metadata:
cluster: shadow-julien
name: connect-1
spec:
displayName: "Connect 1"
urls: "http://localhost:8083"
ignoreUntrustedCertificate: false
headers:
X-PROJECT-HEADER: value
AnotherHeader: test
security:
type: "BasicAuth"
username: "toto"
password: "my-secret"
KafkaConnectCluster checks:
metadata.cluster
must be a valid KafkaCluster namespec.urls
must be a single URL of a Kafka Connect cluster- Multiple URLs are not supported for now. Coming soon
spec.ignoreUntrustedCertificate
(optional, defaultfalse
) must be one of [true
,false
]spec.headers
(optional) must be key-value pairs of HTTP Headersspec.security.type
(optional) must be one of [BasicAuth
,BearerToken
,SSLAuth
]- See HTTP Security Properties for the detailed list of options
KsqlDBCluster
API Keys: Admin API Key
Managed with: API CLI Console UI
Creates a ksqlDB Cluster Definition in Console.
---
apiVersion: console/v2
kind: KsqlDBCluster
metadata:
cluster: julien-cloud
name: ksql-1
spec:
displayName: "KSQL 1"
url: "http://localhost:8088"
ignoreUntrustedCertificate: false
security:
type: "BasicAuth"
username: "toto"
password: "my-secret"
KafkaConnectCluster checks:
metadata.cluster
must be a valid KafkaCluster namespec.url
must be a single URL of a KsqlDB clusterspec.ignoreUntrustedCertificate
(optional, defaultfalse
) must be one of [true
,false
]spec.headers
(optional) must be key-value pairs of HTTP Headersspec.security.type
(optional) must be one of [BasicAuth
,BearerToken
,SSLAuth
]- See HTTP Security Properties for the detailed list of options
Alert
This concept will be available in a future version
DataMaskingPolicy
This concept will be available in a future version
HTTP Security Properties
HTTP Security Properties are used in KafkaCluster (Schema Registry), KafkaConnect, KsqlDBCluster
Basic Authentication
security:
type: "BasicAuth"
username: "toto"
password: "my-secret"
Bearer Token
security:
type: "BearerToken"
token: "toto"
mTLS / Client Certificate
security:
type: "SSLAuth"
key: |
-----BEGIN PRIVATE KEY-----
MIIOXzCCDUegAwIBAgIRAPRytMVYJNUgCbhnA+eYumgwDQYJKoZIhvcNAQELBQAw
...
IFyCs+xkcgvHFtBjjel4pnIET0agtbGJbGDEQBNxX+i4MDA=
-----END PRIVATE KEY-----
certificateChain: |
-----BEGIN CERTIFICATE-----
MIIOXzCCDUegAwIBAgIRAPRytMVYJNUgCbhnA+eYumgwDQYJKoZIhvcNAQELBQAw
RjELMAkGA1UEBhMCVVMxIjAgBgNVBAoTGUdvb2dsZSBUcnVzdCBTZXJ2aWNlcyBM
...
8/s+YDKveNdoeQoAmGQpUmxhvJ9rbNYj+4jiaujkfxT/6WtFN8N95r+k3W/1K4hs
IFyCs+xkcgvHFtBjjel4pnIET0agtbGJbGDEQBNxX+i4MDA=
-----END CERTIFICATE-----
Permissions
Permissions are used in Groups and Users and lets you configure all the access to any Kafka resource or Console feature.
A permission applies to a certain resourceType
, which affect the necessary fields as detailed below.
- Topic Permissions
- Subject Permissions
- ConsumerGroup Permissions
- Cluster Permissions
- KafkaConnect Permissions
- KsqlDB Permissions
- Platform Permissions
Topic Permissions
# Grants Consume, Produce and View Config to all topics toto-* on shadow-it cluster
- resourceType: TOPIC
cluster: shadow-it
patternType: PREFIXED
name: toto-
permissions:
- topicViewConfig
- topicConsume
- topicProduce
resourceType
:TOPIC
cluster
is a valid Kafka clusterpatternType
is eitherPREFIXED
orLITERAL
name
is the name of the topic or topic prefix to apply the permissions topermissions
is a list of valid topic permissions (See Table)
Available Topic Permissions | Description |
---|---|
topicConsume | Permission to consume messages from the topic. |
topicProduce | Permission to produce (write) messages to the topic. |
topicViewConfig | Permission to view the topic configuration. |
topicEditConfig | Permission to edit the topic configuration. |
topicCreate | Permission to create a new topic. |
topicDelete | Permission to delete the topic. |
topicAddPartition | Permission to add partitions to the topic. |
topicEmpty | Permission to empty (delete all messages from) the topic. |
Subject Permissions
# Grants View and Edit Compatibility to all subjects starting with sub-* on shadow-it cluster
- resourceType: SUBJECT
cluster: shadow-it
patternType: PREFIXED
name: sub-
permissions:
- subjectView
- subjectEditCompatibility
resourceType
:SUBJECT
cluster
is a valid Kafka clusterpatternType
is eitherPREFIXED
orLITERAL
name
is the name of the subject or subject prefix to apply the permissions topermissions
is a list of valid subject permissions (See Table)
Available Subject Permissions | Description |
---|---|
subjectCreateUpdate | Permission to create or update the subject. |
subjectDelete | Permission to delete the subject. |
subjectEditCompatibility | Permission to edit the subject compatibility settings. |
subjectView | Permission to view the subject details. |
ConsumerGroup Permissions
# Grants View and Reset on all consumer groups starting with group-* on shadow-it cluster
- resourceType: CONSUMER_GROUP
cluster: shadow-it
patternType: PREFIXED
name: group-
permissions:
- consumerGroupView
- consumerGroupReset
resourceType
:CONSUMER_GROUP
cluster
is a valid Kafka clusterpatternType
is eitherPREFIXED
orLITERAL
name
is the name of the consumer group or consumer group prefix to apply the permissions topermissions
is a list of valid consumer group permissions (See Table)
Available ConsumerGroup Permissions | Description |
---|---|
consumerGroupCreate | Permission to create a new consumer group. |
consumerGroupReset | Permission to reset the consumer group. |
consumerGroupDelete | Permission to delete the consumer group. |
consumerGroupView | Permission to view the consumer group details. |
Cluster Permissions
# Grants View Broker, Edit Schema Registry Compatibility, Edit Broker, View ACL, and Manage ACL on shadow-it cluster
- resourceType: CLUSTER
name: shadow-it
permissions:
- clusterViewBroker
- clusterEditSRCompatibility
- clusterEditBroker
- clusterViewACL
- clusterManageACL
resourceType
:CLUSTER
name
is the name of the cluster to apply the permissions to- Use
*
for all clusters
- Use
permissions
is a list of valid cluster permissions (See Table)
Available Cluster Permissions | Description |
---|---|
clusterViewBroker | Permission to view broker details. |
clusterEditSRCompatibility | Permission to edit Schema Registry compatibility settings. |
clusterEditBroker | Permission to edit broker configuration. |
clusterViewACL | Permission to view Access Control Lists (ACLs) for the cluster. |
clusterManageACL | Permission to manage Access Control Lists (ACLs) for the cluster. |
KafkaConnect Permissions
# Grants Create and Delete on all connectors starting with connector-* on shadow-it cluster and kafka-connect-cluster
- resourceType: KAFKA_CONNECT
cluster: shadow-it
kafkaConnect: kafka-connect-cluster
patternType: PREFIXED
name: connector-
permissions:
- kafkaConnectorCreate
- kafkaConnectorDelete
resourceType
:KAFKA_CONNECT
cluster
is a valid Kafka clusterkafkaConnect
is a valid Kafka Connect clusterpatternType
is eitherPREFIXED
orLITERAL
name
is the name of the connector or connector prefix to apply the permissions topermissions
is a list of valid Kafka Connect permissions (See Table)
Available KafkaConnect Permissions | Description |
---|---|
kafkaConnectorViewConfig | Permission to view the Kafka Connect configuration. |
kafkaConnectorStatus | Permission to view the status of Kafka Connect connectors. |
kafkaConnectorEditConfig | Permission to edit the Kafka Connect configuration. |
kafkaConnectorDelete | Permission to delete connectors. |
kafkaConnectorCreate | Permission to create new connectors. |
kafkaConnectPauseResume | Permission to pause and resume connectors. |
kafkaConnectRestart | Permission to restart connectors. |
KsqlDB Permissions
# Grants all permissions on KsqlDB cluster ksql-cluster
- resourceType: KSQLDB
cluster: shadow-it
ksqlDB: ksql-cluster
permissions:
- ksqldbAccess
resourceType
:KSQLDB
cluster
is a valid Kafka clusterksqlDB
is a valid Kafka Connect clusterpermissions
is a list of valid KsqlDB permissions (See Table)
Available KafkaConnect Permissions | Description |
---|---|
ksqldbAccess | Grants all permissions on the KsqlDB Cluster. |
Platform Permissions
# Grants Platform permissions
- resourceType: PLATFORM
permissions:
- userView
- datamaskingView
resourceType
:PLATFORM
permissions
is a list of valid Platform permissions (See Table)
Available KafkaConnect Permissions | Description |
---|---|
clusterConnectionsManage | Permission to add / edit / remove Kafka clusters on Console |
certificateManage | Permission to add / edit / remove TLS Certificates on Console |
userManage | Permission to manage Console users, groups & permissions |
userView | Permission to view Console users, groups & permissions |
datamaskingManage | Permission to manage Data policies (masking rules) |
datamaskingView | Permission to view Data policies |
notificationChannelManage | Permission to manage Integration channels |
notificationChannelView | Permission to view Integration channels |
auditLogView | Permission to browse audit log |