Skip to main content
These are high-level Gateway object specifications for Conduktor CLI.

​
Deploy Interceptor

Deploys an Interceptor on Gateway.
  • Managed with: API, CLI, UI, TF
---
apiVersion: gateway/v2
kind: Interceptor
metadata:
  name: enforce-partition-limit
  # scope:
  #   vCluster: aaa
  #   group: bbb
  #   username: ccc
spec:
  pluginClass: "io.conduktor.gateway.interceptor.safeguard.CreateTopicPolicyPlugin"
  priority: 100
  config:
    topic: "myprefix-.*"
    numPartition:
      min: 5
      max: 5
      action: "INFO"
Interceptor checks:
  • metadata.scope is optional (default empty).
  • metadata.scope.[vCluster | group | username] combine with each other to define the targeting
  • spec.pluginClass is mandatory. Has to be a valid Interceptor class name.
  • spec.priority is mandatory
  • spec.config is a valid config for the pluginClass

​
Interceptor targeting

You can activate your Interceptor only in specific scenarios. Use the table below to configure Targeting settings.
Use casemetadata.scope.vclustermetadata.scope.groupmetadata.scope.username
Global Interceptor (Including Virtual Clusters)Set to nullSet to nullSet to null
Global Interceptor (Excluding Virtual Clusters)EmptyEmptyEmpty
Username targetingEmptyEmptySet
Group targetingEmptySetEmpty
Virtual Cluster targetingSetEmptyEmpty
Virtual Cluster + Username targetingSetEmptySet
Virtual Cluster + Group targetingSetSetEmpty
You can deploy multiple Interceptors with the same name using a different targeting scope. This will effectively override the configuration for the scope.
The order of precedence from highest (overrides all others) to lowest (most easily overridden) is:
  • ServiceAccount
  • Group
  • VirtualCluster
  • Global

​
Examples

  ---
  # This Interceptor targets everyone (Including Virtual Clusters)
  apiVersion: gateway/v2
  kind: Interceptor
  metadata:
    name: enforce-partition-limit
    scope:
      vCluster: null
      group: null
      username: null
  spec:

  ---
  # This Interceptor targets everyone (Excluding Virtual Clusters)
  apiVersion: gateway/v2
  kind: Interceptor
  metadata:
    name: enforce-partition-limit
  spec:

  ---
  # This Interceptor targets only `admin` service account
  apiVersion: gateway/v2
  kind: Interceptor
  metadata:
    name: enforce-partition-limit
    scope:
      username: admin
  spec:

  ---
  # This interceptor targets only `read-only` virtual cluster
  apiVersion: gateway/v2
  kind: Interceptor
  metadata:
    name: enforce-partition-limit
    scope:
      vCluster: read-only
  spec:

​
Supported authentication methods

Gateway supports different client authentication methods depending on the security mode (Gateway-managed or Kafka-managed) and service account type (local or external).
ModeProtocolLocal service accountExternal service account
Anonymous
GATEWAY_MANAGEDPLAINTEXT❌❌
SSL❌❌
SSL with client auth (mTLS)❌✅
SSL❌✅
SASL
SASL_PLAINTEXT✅only if OAUTHBEARER
SASL_SSL✅only if OAUTHBEARER
KAFKA_MANAGEDSASL_PLAINTEXT❌✅
SASL_SSL❌✅
As of Gateway v3.10.0, the DELEGATED_XXX security protocols have been deprecated in favor of the security mode, set by the GATEWAY_SECURITY_MODE environment variable. These values remain supported for backward compatibility but will be deprecated in future releases.If you’re using DELEGATED security protocols, see the security mode migration guide.

​
Authentication method details

Anonymous authentication
  • PLAINTEXT: Client is anonymous, no credentials needed. Local and external service accounts not supported.
  • SSL (encryption only): Transport encryption without client authentication. Local and external service accounts not supported.
SSL with client authentication (mTLS)
  • Both Gateway and clients validate each other’s identities using TLS certificates
  • Gateway extracts user identity from the TLS certificate
  • Identity can be mapped to an external service account in Gateway
  • Username format: CN=writeuser,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown
  • Customize with GATEWAY_SSL_PRINCIPAL_MAPPING_RULES environment variable
SASL authentication
  • With OAUTHBEARER: clients authenticate with an identity (the sub in the OIDC JWT token)
  • Can be mapped to an external service account in Gateway
  • Supports PLAIN and SCRAM mechanisms in Kafka-managed mode
  • JWT token required with grant type clientcredentials for OAUTHBEARER
Kafka-managed SASL authentication
  • Delegates authentication to the backing Kafka cluster
  • Gateway forwards client credentials to Kafka for authentication
  • Kafka cluster retrieves ACL rules and authenticates the client
  • Supports PLAIN, SCRAM, OAUTHBEARER, and AWS_MSK_IAM mechanisms
  • External service accounts can still be mapped for friendly names in Gateway
  • Local service accounts not available in this mode
  • Virtual resources (Virtual Clusters, alias topics, concentrated topics) not available

​
What Gateway manages

Conduktor Gateway manages service accounts, client authentication and ACLs (Access Control Lists) by determining:
  • where to authenticate clients, Gateway or Kafka
  • which authentication method to use
  • when to use local or external service accounts
  • how and where to manage service account ACLs

​
Troubleshooting authentication and authorization

No, Conduktor’s RBAC model is designed to manage user permissions within Conduktor Console, specifically for interacting with Kafka resources through Conduktor’s interface. However, using Conduktor’s Self-service, you can create Access Control Lists (ACLs) based on applications. This allows application teams to manage their Kafka resources autonomously, ensuring that each application has the necessary permissions to interact with Kafka topics, consumer groups and other resources.
Yes, Conduktor is fully compatible with IAM for MSK authentication but doesn’t currently support modifying IAM policies.
Find out more about Gateway authentication and authorization

​
GatewayServiceAccount

When using Oauth, mTLS or delegated backing Kafka authentication, GatewayServiceAccount is generally optional.
  • Managed with: API, CLI, TF
Each service account is stored in an internal topic, _conduktor_${GATEWAY_CLUSTER_ID}_usermappings and includes a name (used when applying ACLs and Interceptors) and an associated Virtual Cluster. By default, this is set to the default Virtual Cluster, passthrough.
When working with external service accounts, it’s important to understand the difference between:
  • Internal name (metadata.name): the friendly name you assign in Gateway - use this in Interceptor scopes, ACLs, audit logs
  • External name (spec.externalNames): the original identifier from the identity provider, only used for mapping external identities.
GatewayServiceAccount resource is enabled/disabled depending on your Gateway configuration. This is to prevent you from declaring a resource that’s incompatible with your current configuration:
Security Mode & ProtocolLOCAL GatewayServiceAccountEXTERNAL GatewayServiceAccount
GATEWAY_MANAGED + PLAINTEXT🚫🚫
GATEWAY_MANAGED + SSL🚫only if mTLS configured
GATEWAY_MANAGED + SASL_PLAINTEXT✅only if OAuth configured
GATEWAY_MANAGED + SASL_SSL✅only if OAuth configured
KAFKA_MANAGED + SASL_PLAINTEXT🚫✅
KAFKA_MANAGED + SASL_SSL🚫✅
DELEGATED_SASL_PLAINTEXT (deprecated, use KAFKA_MANAGED)🚫✅
DELEGATED_SASL_SSL (deprecated, use KAFKA_MANAGED)🚫✅
Here are a few cases where you have to declare GatewayServiceAccount objects:
  • creating local service accounts
  • renaming service accounts for easier clarity when using Interceptors
  • attaching service accounts to Virtual Clusters
---
# External user renamed
apiVersion: gateway/v2
kind: GatewayServiceAccount
metadata:
  name: application1
spec:
  type: EXTERNAL
  externalNames:
  - 00u9vme99nxudvxZA0h7
---
# Local User on Virtual Cluster vc-B
apiVersion: gateway/v2
kind: GatewayServiceAccount
metadata:
  vCluster: vc-B
  name: admin
spec:
  type: LOCAL
GatewayServiceAccount checks:
  • When spec.type is EXTERNAL:
    • spec.externalNames must be a non-empty list of external names. Each name must be unique across all declared GatewayServiceAccount.
    • we currently only support a list of one element.
GatewayServiceAccount side effects:
  • When spec.type is EXTERNAL:
    • During Client connection, the authenticated user will be checked against the list of externalNames to decide which GatewayServiceAccount it is.
  • When spec.type is LOCAL:
    • Access to /gateway/v2/tokens endpoint to generate a password for this service account
    • Switching a GatewayServiceAccount spec.type from LOCAL to EXTERNAL does not invalidate previously emitted tokens. They will keep on working for their TTL.

​
Gateway service account groups

Application service accounts defined in Gateway that follow a common set of Interceptor rules can be grouped. This allows you to scope Interceptors for multiple service accounts.
These groups can’t be used for managing ACLs of the service accounts.

​
Create a service account group

Use the Gateway API to create a group. See the GatewayGroup section below for more details.

​
Apply an Interceptor to a group

Once a group is created, you can apply Interceptors to it directly from within the Interceptor configuration. Use metadata.scope.group to define which group the Interceptor should apply to. See the Interceptor targeting section above for more details.

​
GatewayGroup

Gateway group lets you add multiple users in the same GatewayGroup for easier Interceptor targeting capabilities. 
---
# Users added to the group manually
apiVersion: gateway/v2
kind: GatewayGroup
metadata:
  name: group-a
spec:
  members:
    - name: admin
    - vCluster: vc-B
      name: "0000-AAAA-BBBB-CCCC"
GatewayGroup checks:
  • spec.members[].name is mandatory.
    • Currently, the username needs to refer to an existing GatewayServiceAccount otherwise it will fail. This is a known issue that we’ll address in a further release.
  • spec.members[].vCluster is optional. It has to refer to an existing Virtual Cluster. When not using Virtual Clusters, don’t set this attribute.
GatewayGroup side effects:
  • All members of the group will be affected by Interceptors deployed with this group’s scope.

​
ConcentrationRule

Concentration Rules allow you to define patterns where topic creation won’t generate a physical topic, but will instead use our topic concentration feature. 
---
apiVersion: gateway/v2
kind: ConcentrationRule
metadata:
  # vCluster: vc-B
  name: toutdanstiti
spec:
  pattern: titi-.*
  physicalTopics:
    delete: titi-delete
    compact: titi-compact
    deleteCompact: titi-cd
  autoManaged: false
  offsetCorrectness: false
ConcentrationRule checks:
  • metadata.vCluster is optional. Must refer to an existing Virtual Cluster. When not using Virtual Clusters, don’t set this attribute.
  • spec.physicalTopics.delete is mandatory. Has to be a valid topic name with a cleanup.policy set to delete.
  • spec.physicalTopics.compact is optional. Has ti be a valid topic name with a cleanup.policy set to compact.
  • spec.physicalTopics.deleteCompact is optional. Has to be a valid topic name with a cleanup.policy set to delete,compact.
  • spec.autoManaged is optional, default is false.
  • spec.offsetCorrectness is optional, default is false.
ConcentrationRule side effects:
  • Once the Concentration Rule is deployed, topics created with a name matching the spec.pattern will not be created as real Kafka topics but as concentrated topics instead.
  • Depending on the topic’s cleanup.policy, the topic’s data will be stored in one of the configured physical topics.
  • If a topic creation request is made with a cleanup.policy that isn’t configured in the ConcentrationRule, topic creation will fail.
  • It is not possible to update cleanup.policy of a concentrated topic.
  • If spec.autoManaged is set to true, the underlying physical topics and configurations will be automatically created and/or extended to honour the topics configurations.
  • If spec.offsetCorrectness is set to true, Gateway will maintain a list of offsets for each of the concentrated topic records.
    • This allows for a proper calculation of message count and consumer group lag.
    • There are some limitations with offset correctness.
  • If spec.offsetCorrectness is set to false, Gateway will report the offsets of the backing topic records.
If a ConcentrationRule spec changes, it will not affect previously created concentrated topics, it will only affect the topics created after the change.

​
VirtualCluster

A Virtual Cluster creates an isolated Kafka environment within Gateway. Topics and consumer groups in one Virtual Cluster are completely separate from other Virtual Clusters which enables multi-tenancy on a single physical cluster.
  • Managed with: API, CLI
---
apiVersion: gateway/v2
kind: VirtualCluster
metadata:
  name: customers
spec:
  type: Standard
  aclEnabled: true
  aclMode: KAFKA_API
  superUsers:
    - admin
VirtualCluster fields:
​
metadata.name
string
required
The unique identifier for the Virtual Cluster and serves as the prefix for all topics and consumer groups on the physical Kafka clusterPattern: ^[a-zA-Z0-9._-]+$ (letters, numbers, dots, underscores, hyphens only)Impact: all topics and consumer groups created in this Virtual Cluster are prefixed with this name on the physical cluster. They appear in the Virtual Cluster without the prefix.
  • Default behavior: creating a topic orders in the customers Virtual Cluster results in a physical topic called customersorders
  • Automatic discovery: the physical topic customerscatalog automatically appears as catalog in the customers Virtual Cluster
  • Override with alias mappings: use AliasTopic to map logical topics to any physical topic name, regardless of the prefix
​
spec.aclEnabled
boolean
default:"false"
Controls authorization for the Virtual Cluster
  • When false: No access control. Clients can create, delete, produce, consume, and manage all topics and consumer groups in this Virtual Cluster. The fields acls and superUsers can’t be set.
  • When true: requires ACL configuration (see sections below)
​
spec.aclMode
string
Determines how ACLs are managed. When omitted:
  • if aclEnabled: true: defaults to KAFKA_API
  • if aclEnabled: false: remains unset (no ACL mode)
Once aclMode is set, it can’t be changed. The KAFKA_API and REST_API modes are incompatible (cumulative vs idempotent operations).
​
spec.type
string
default:"Standard"
Defines the Virtual Cluster type
  • Standard: For internal multi-tenancy within your organization
  • Partner: For external data sharing with external organizations. When using aclMode: KAFKA_API, ACLs must be enabled with at least one superUser
The next two sections describe the behaviour of spec.aclEnabled when it is set to true
Manage ACLs using the Kafka Admin API, just like you would with a standard Kafka cluster. Administrators listed in superUsers can dynamically create, update, and delete ACLs using Kafka CLI tools.
---
apiVersion: gateway/v2
kind: VirtualCluster
metadata:
  name: customers
spec:
  type: Standard
  aclEnabled: true
  aclMode: KAFKA_API
  superUsers:
    - admin
Requirements
  • at least one spec.superUsers is required: a list of usernames whose associated Gateway Service Accounts can bypass ACLs. These have to be created separately.
  • not allowed: the spec.acls field
Manage ACLs declaratively via the Gateway REST API. All ACL bindings are defined in the Virtual Cluster YAML configuration, enabling Infrastructure-as-Code workflows.
---
apiVersion: gateway/v2
kind: VirtualCluster
metadata:
  name: customers
spec:
  type: Standard
  aclEnabled: true
  aclMode: REST_API
  acls:
    - resourcePattern:
        resourceType: TOPIC
        name: orders
        patternType: LITERAL
      principal: User:orders-app
      host: "*"
      operation: READ
      permissionType: ALLOW
    - resourcePattern:
        resourceType: TOPIC
        name: orders
        patternType: LITERAL
      principal: User:orders-app
      host: "*"
      operation: WRITE
      permissionType: ALLOW
Requirements:
  • at least one spec.acls is required: a complete list of ACL bindings for the Virtual Cluster.
    • any valid Kafka API ACL binding including * wildcards is supported. For a complete reference, see the API schema
  • not allowed: the spec.superUsers field

​
AliasTopic

An Alias Topic maps a physical Kafka topic to a logical name within a Virtual Cluster. You can use it to:
  • Expose internal topics to external partners with business-friendly names while keeping your internal naming conventions private
  • Map a logical topic in a Virtual Cluster to any physical topic, regardless of the Virtual Cluster’s prefix
  • Share the same physical topic across multiple Virtual Clusters with different logical names
  • Managed with: API, CLI 
---
apiVersion: gateway/v2
kind: AliasTopic
metadata:
  name: stock                              # Logical name exposed to partner
  vCluster: external-partner               # Partner's isolated environment
spec:
  physicalName: internal.manufacturing.eu.stock  # Internal topic with sensitive naming
  physicalCluster: main                    # Physical cluster (defaults to main)
In this example, the partner sees a topic called stock in their Virtual Cluster, while it actually maps to your internal topic internal.manufacturing.eu.stock that contains European manufacturing stock data. The internal naming convention (region, department structure) remains hidden from the partner. AliasTopic fields:
​
metadata.name
string
required
The logical topic name visible in the Virtual ClusterPattern: ^[a-zA-Z0-9._-]+$ (letters, numbers, dots, underscores, hyphens only)
​
metadata.vCluster
string
default:"passthrough"
Target Virtual Cluster for this aliasPattern: ^[a-zA-Z0-9._-]+$ (letters, numbers, dots, underscores, hyphens only)
​
spec.physicalName
string
required
The actual physical topic name on KafkaPattern: ^[a-zA-Z0-9._-]+$ (letters, numbers, dots, underscores, hyphens only)Can be any valid topic name, no prefix requirement
​
spec.physicalCluster
string
Physical Kafka cluster hosting the topic
  • For Partner Virtual Clusters: required, must be explicitly specified
  • For Standard Virtual Clusters: optional, defaults to main (only valid value)
  • Must match one of the Gateway’s configured physical clusters