> ## Documentation Index > Fetch the complete documentation index at: https://docs.conduktor.io/llms.txt > Use this file to discover all available pages before exploring further. # Gateway resources > YAML resource definitions for Conduktor Gateway: Virtual Clusters, service accounts, Interceptors, and authentication configuration schemas with examples. These are high-level Gateway object specifications for Conduktor CLI. ## Deploy Interceptor Deploys an Interceptor on Gateway. * **Managed with:** API, CLI, UI, TF ```yaml theme={null} --- apiVersion: gateway/v2 kind: Interceptor metadata: name: enforce-partition-limit # scope: # vCluster: aaa # group: bbb # username: ccc spec: pluginClass: "io.conduktor.gateway.interceptor.safeguard.CreateTopicPolicyPlugin" priority: 100 config: topic: "myprefix-.*" numPartition: min: 5 max: 5 action: "INFO" ``` **Interceptor checks:** * `metadata.scope` is optional (default empty). * `metadata.scope.[vCluster | group | username]` combine with each other to define the targeting * Check the dedicated [Interceptor targeting](#interceptor-targeting) section * `spec.pluginClass` is **mandatory**. Has to be a valid Interceptor class name. * `spec.priority` is **mandatory** * `spec.config` is a valid config for the `pluginClass` [View Terraform documentation](https://registry.terraform.io/providers/conduktor/conduktor/latest/docs/resources/gateway_interceptor_v2) ### Interceptor targeting You can activate your Interceptor only in specific scenarios. Use the table below to configure Targeting settings. | Use case | `metadata.scope.vcluster` | `metadata.scope.group` | `metadata.scope.username` | | --------------------------------------------------- | ------------------------- | ---------------------- | ------------------------- | | Global Interceptor (Including Virtual Clusters) | Set to `null` | Set to `null` | Set to `null` | | Global Interceptor (**Excluding** Virtual Clusters) | Empty | Empty | Empty | | Username targeting | Empty | Empty | Set | | Group targeting | Empty | Set | Empty | | Virtual Cluster targeting | Set | Empty | Empty | | Virtual Cluster + Username targeting | Set | Empty | Set | | Virtual Cluster + Group targeting | Set | Set | Empty | You can deploy multiple Interceptors with the same name using a different targeting scope. This will effectively override the configuration for the scope. The order of precedence from highest (overrides all others) to lowest (most easily overridden) is: * ServiceAccount * Group * VirtualCluster * Global #### Examples ```yaml theme={null} --- # This Interceptor targets everyone (Including Virtual Clusters) apiVersion: gateway/v2 kind: Interceptor metadata: name: enforce-partition-limit scope: vCluster: null group: null username: null spec: --- # This Interceptor targets everyone (Excluding Virtual Clusters) apiVersion: gateway/v2 kind: Interceptor metadata: name: enforce-partition-limit spec: --- # This Interceptor targets only `admin` service account apiVersion: gateway/v2 kind: Interceptor metadata: name: enforce-partition-limit scope: username: admin spec: --- # This interceptor targets only `read-only` virtual cluster apiVersion: gateway/v2 kind: Interceptor metadata: name: enforce-partition-limit scope: vCluster: read-only spec: ``` ## Supported authentication methods Gateway supports different client authentication methods depending on the security mode (Gateway-managed or Kafka-managed) and service account type (local or external). | **Mode** | **Protocol** | [Local service account](/guide/conduktor-concepts/gateway-service-accounts/#local-service-accounts) | [External service account](/guide/conduktor-concepts/gateway-service-accounts/#external-service-accounts) | | ----------------- | ------------------------------- | :-------------------------------------------------------------------------------------------------: | :-------------------------------------------------------------------------------------------------------: | | | **Anonymous** | | | | `GATEWAY_MANAGED` | PLAINTEXT | ❌ | ❌ | | | SSL | ❌ | ❌ | | | **SSL with client auth (mTLS)** | ❌ | ✅ | | | SSL | ❌ | ✅ | | | **SASL** | | | | | SASL\_PLAINTEXT | ✅ | only if `OAUTHBEARER` | | | SASL\_SSL | ✅ | only if `OAUTHBEARER` | | `KAFKA_MANAGED` | SASL\_PLAINTEXT | ❌ | ✅ | | | SASL\_SSL | ❌ | ✅ | As of Gateway v3.10.0, the `DELEGATED_XXX` security protocols have been deprecated in favor of the security mode, set by the `GATEWAY_SECURITY_MODE` environment variable. These values remain supported for backward compatibility but will be deprecated in future releases. If you're using `DELEGATED` security protocols, see the [security mode migration guide](/guide/tutorials/migrate-gateway-security). ### Authentication method details **Anonymous authentication** * PLAINTEXT: Client is anonymous, no credentials needed. Local and external service accounts not supported. * SSL (encryption only): Transport encryption without client authentication. Local and external service accounts not supported. **SSL with client authentication (mTLS)** * Both Gateway and clients validate each other's identities using TLS certificates * Gateway extracts user identity from the TLS certificate * Identity can be mapped to an external service account in Gateway * Username format: `CN=writeuser,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown` * Customize with `GATEWAY_SSL_PRINCIPAL_MAPPING_RULES` environment variable **SASL authentication** * With OAUTHBEARER: clients authenticate with an identity (the `sub` in the OIDC JWT token) * Can be mapped to an external service account in Gateway * Supports PLAIN and SCRAM mechanisms in Kafka-managed mode * JWT token required with grant type `clientcredentials` for OAUTHBEARER **Kafka-managed SASL authentication** * Delegates authentication to the backing Kafka cluster * Gateway forwards client credentials to Kafka for authentication * Kafka cluster retrieves ACL rules and authenticates the client * Supports PLAIN, SCRAM, OAUTHBEARER, and AWS\_MSK\_IAM mechanisms * External service accounts can still be mapped for friendly names in Gateway * Local service accounts not available in this mode * Virtual resources (Virtual Clusters, alias topics, concentrated topics) not available ### What Gateway manages Conduktor Gateway manages service accounts, client authentication and ACLs (Access Control Lists) by determining: * **where** to authenticate clients, Gateway or Kafka * which **authentication method** to use * **when** to use local or external service accounts * how and where to manage service account ACLs ### Troubleshooting authentication and authorization No, Conduktor's RBAC model is designed to manage user permissions within Conduktor Console, specifically for interacting with Kafka resources through Conduktor's interface. However, using Conduktor's Self-service, you can create Access Control Lists (ACLs) based on applications. This allows application teams to manage their Kafka resources autonomously, ensuring that each application has the necessary permissions to interact with Kafka topics, consumer groups and other resources. Yes, Conduktor is fully compatible with IAM for MSK authentication but doesn't currently support modifying IAM policies. [Find out more about Gateway authentication and authorization](/guide/conduktor-concepts/gateway-authentication-authorization) ## GatewayServiceAccount When using **Oauth**, **mTLS** or delegated backing Kafka authentication, `GatewayServiceAccount` is generally optional. * **Managed with:** API, CLI, TF Each service account is stored in an internal topic, `_conduktor_${GATEWAY_CLUSTER_ID}_usermappings` and includes a **name** (used when applying ACLs and Interceptors) and an associated Virtual Cluster. By default, this is set to the default Virtual Cluster, `passthrough`. When working with external service accounts, it's important to understand the difference between: * **Internal name** (`metadata.name`): the friendly name you assign in Gateway - use this in Interceptor scopes, ACLs, audit logs * **External name** (`spec.externalNames`): the original identifier from the identity provider, only used for mapping external identities. GatewayServiceAccount resource is enabled/disabled depending on your Gateway configuration. This is to prevent you from declaring a resource that's incompatible with your current configuration: | Security Mode & Protocol | LOCAL GatewayServiceAccount | EXTERNAL GatewayServiceAccount | | ------------------------------------------------------------- | --------------------------- | ------------------------------ | | `GATEWAY_MANAGED` + `PLAINTEXT` | 🚫 | 🚫 | | `GATEWAY_MANAGED` + `SSL` | 🚫 | only if mTLS configured | | `GATEWAY_MANAGED` + `SASL_PLAINTEXT` | ✅ | only if OAuth configured | | `GATEWAY_MANAGED` + `SASL_SSL` | ✅ | only if OAuth configured | | `KAFKA_MANAGED` + `SASL_PLAINTEXT` | 🚫 | ✅ | | `KAFKA_MANAGED` + `SASL_SSL` | 🚫 | ✅ | | `DELEGATED_SASL_PLAINTEXT` *(deprecated, use KAFKA\_MANAGED)* | 🚫 | ✅ | | `DELEGATED_SASL_SSL` *(deprecated, use KAFKA\_MANAGED)* | 🚫 | ✅ | Here are a few cases where **you have to** declare GatewayServiceAccount objects: * creating local service accounts * renaming service accounts for easier clarity when using Interceptors * attaching service accounts to Virtual Clusters ```yaml theme={null} --- # External user renamed apiVersion: gateway/v2 kind: GatewayServiceAccount metadata: name: application1 spec: type: EXTERNAL externalNames: - 00u9vme99nxudvxZA0h7 --- # Local User on Virtual Cluster vc-B apiVersion: gateway/v2 kind: GatewayServiceAccount metadata: vCluster: vc-B name: admin spec: type: LOCAL ``` **GatewayServiceAccount checks:** * When `spec.type` is `EXTERNAL`: * `spec.externalNames` must be a non-empty list of external names. Each name must be unique across all declared GatewayServiceAccount. * we currently only support a list of one element. [View Terraform documentation](https://registry.terraform.io/providers/conduktor/conduktor/latest/docs/resources/gateway_service_account_v2) **GatewayServiceAccount side effects:** * When `spec.type` is `EXTERNAL`: * During Client connection, the authenticated user will be checked against the list of `externalNames` to decide which GatewayServiceAccount it is. * When `spec.type` is `LOCAL`: * Access to `/gateway/v2/tokens` endpoint to generate a password for this service account * Switching a GatewayServiceAccount `spec.type` from `LOCAL` to `EXTERNAL` does not invalidate previously emitted tokens. They will keep on working for their TTL. ### Gateway service account groups Application service accounts defined in Gateway that follow a common set of Interceptor rules can be grouped. This allows you to scope Interceptors for multiple service accounts. These groups can't be used for managing ACLs of the service accounts. #### Create a service account group Use the Gateway API to create a `group`. See the GatewayGroup section below for more details. #### Apply an Interceptor to a group Once a group is created, you can apply Interceptors to it directly from within the Interceptor configuration. Use `metadata.scope.group` to define which group the Interceptor should apply to. See the Interceptor targeting section above for more details. ## GatewayGroup Gateway group lets you add multiple users in the same GatewayGroup for easier Interceptor targeting capabilities. ```yaml theme={null} --- # Users added to the group manually apiVersion: gateway/v2 kind: GatewayGroup metadata: name: group-a spec: members: - name: admin - vCluster: vc-B name: "0000-AAAA-BBBB-CCCC" ``` **GatewayGroup checks:** * `spec.members[].name` is mandatory. * Currently, the username needs to refer to an existing GatewayServiceAccount otherwise it will fail. This is a known issue that we'll address in a further release. * `spec.members[].vCluster` is optional. It has to refer to an existing Virtual Cluster. When not using Virtual Clusters, don't set this attribute. **GatewayGroup side effects:** * All members of the group will be affected by Interceptors deployed with this group's scope. ## ConcentrationRule Concentration Rules allow you to define patterns where topic creation won't generate a physical topic, but will instead use our topic concentration feature. ```yaml theme={null} --- apiVersion: gateway/v2 kind: ConcentrationRule metadata: # vCluster: vc-B name: toutdanstiti spec: pattern: titi-.* physicalTopics: delete: titi-delete compact: titi-compact deleteCompact: titi-cd autoManaged: false offsetCorrectness: false ``` **ConcentrationRule checks:** * `metadata.vCluster` is optional. Must refer to an existing Virtual Cluster. When not using Virtual Clusters, don't set this attribute. * `spec.physicalTopics.delete` is mandatory. Has to be a valid topic name with a `cleanup.policy` set to `delete`. * `spec.physicalTopics.compact` is optional. Has ti be a valid topic name with a `cleanup.policy` set to `compact`. * `spec.physicalTopics.deleteCompact` is optional. Has to be a valid topic name with a `cleanup.policy` set to `delete,compact`. * `spec.autoManaged` is optional, default is `false`. * `spec.offsetCorrectness` is optional, default is `false`. **ConcentrationRule side effects:** * Once the Concentration Rule is deployed, topics created with a name matching the `spec.pattern` will not be created as real Kafka topics but as concentrated topics instead. * Depending on the topic's `cleanup.policy`, the topic's data will be stored in one of the configured physical topics. * If a topic creation request is made with a `cleanup.policy` that isn't configured in the ConcentrationRule, topic creation will fail. * It is not possible to update `cleanup.policy` of a concentrated topic. * If `spec.autoManaged` is set to `true`, the underlying physical topics and configurations will be automatically created and/or extended to honour the topics configurations. * If `spec.offsetCorrectness` is set to `true`, Gateway will maintain a list of offsets for each of the concentrated topic records. * This allows for a proper calculation of message count and consumer group lag. * There are some limitations with offset correctness. * If `spec.offsetCorrectness` is set to `false`, Gateway will report the offsets of the backing topic records. If a *ConcentrationRule* spec changes, it will not affect previously created concentrated topics, it will only affect the topics created after the change. ## VirtualCluster A Virtual Cluster creates an isolated Kafka environment within Gateway. Topics and consumer groups in one Virtual Cluster are completely separate from other Virtual Clusters which enables multi-tenancy on a single physical cluster. * **Managed with:** API, CLI ```yaml theme={null} --- apiVersion: gateway/v2 kind: VirtualCluster metadata: name: customers spec: type: Standard aclEnabled: true aclMode: KAFKA_API superUsers: - admin ``` **VirtualCluster fields:** The unique identifier for the Virtual Cluster and serves as the prefix for all topics and consumer groups on the physical Kafka cluster Pattern: `^[a-zA-Z0-9._-]+$` (letters, numbers, dots, underscores, hyphens only) **Impact**: all topics and consumer groups created in this Virtual Cluster are prefixed with this name on the physical cluster. They appear in the Virtual Cluster without the prefix. * **Default behavior**: creating a topic `orders` in the `customers` Virtual Cluster results in a physical topic called `customersorders` * **Automatic discovery**: the physical topic `customerscatalog` automatically appears as `catalog` in the `customers` Virtual Cluster * **Override with alias mappings**: use [AliasTopic](#aliastopic) to map logical topics to any physical topic name, regardless of the prefix Controls authorization for the Virtual Cluster * When `false`: No access control. Clients can create, delete, produce, consume, and manage all topics and consumer groups in this Virtual Cluster. The fields `acls` and `superUsers` can't be set. * When `true`: requires ACL configuration (see sections below) Determines how ACLs are managed. When omitted: * if `aclEnabled: true`: defaults to `KAFKA_API` * if `aclEnabled: false`: remains unset (no ACL mode) Once `aclMode` is set, it can't be changed. The `KAFKA_API` and `REST_API` modes are incompatible (cumulative vs idempotent operations). Defines the Virtual Cluster type * **Standard**: For internal multi-tenancy within your organization * **Partner**: For external data sharing with external organizations. When using `aclMode: KAFKA_API`, ACLs must be enabled with at least one superUser The next two sections describe the behaviour of `spec.aclEnabled` when it is set to `true` Manage ACLs using the Kafka Admin API, just like you would with a standard Kafka cluster. Administrators listed in `superUsers` can dynamically create, update, and delete ACLs using Kafka CLI tools. ```yaml theme={null} --- apiVersion: gateway/v2 kind: VirtualCluster metadata: name: customers spec: type: Standard aclEnabled: true aclMode: KAFKA_API superUsers: - admin ``` **Requirements** * at least one `spec.superUsers` is required: a list of usernames whose associated [Gateway Service Accounts](#gatewayserviceaccount) can bypass ACLs. These **have to be created separately**. * not allowed: the `spec.acls` field Manage ACLs declaratively via the Gateway REST API. All ACL bindings are defined in the Virtual Cluster YAML configuration, enabling Infrastructure-as-Code workflows. ```yaml theme={null} --- apiVersion: gateway/v2 kind: VirtualCluster metadata: name: customers spec: type: Standard aclEnabled: true aclMode: REST_API acls: - resourcePattern: resourceType: TOPIC name: orders patternType: LITERAL principal: User:orders-app host: "*" operation: READ permissionType: ALLOW - resourcePattern: resourceType: TOPIC name: orders patternType: LITERAL principal: User:orders-app host: "*" operation: WRITE permissionType: ALLOW ``` **Requirements:** * at least one `spec.acls` is required: a complete list of ACL bindings for the Virtual Cluster. * any valid Kafka API ACL binding including `*` wildcards is supported. For a complete reference, [see the API schema](https://developers.conduktor.io/?product=gateway) * not allowed: the `spec.superUsers` field ## AliasTopic An Alias Topic maps a physical Kafka topic to a logical name within a Virtual Cluster. You can use it to: * Expose internal topics to external partners with business-friendly names while keeping your internal naming conventions private * Map a logical topic in a Virtual Cluster to any physical topic, regardless of the Virtual Cluster's prefix * Share the same physical topic across multiple Virtual Clusters with different logical names * **Managed with:** API, CLI ```yaml theme={null} --- apiVersion: gateway/v2 kind: AliasTopic metadata: name: stock # Logical name exposed to partner vCluster: external-partner # Partner's isolated environment spec: physicalName: internal.manufacturing.eu.stock # Internal topic with sensitive naming physicalCluster: main # Physical cluster (defaults to main) ``` In this example, the partner sees a topic called `stock` in their Virtual Cluster, while it actually maps to your internal topic `internal.manufacturing.eu.stock` that contains European manufacturing stock data. The internal naming convention (region, department structure) remains hidden from the partner. **AliasTopic fields:** The logical topic name visible in the Virtual Cluster Pattern: `^[a-zA-Z0-9._-]+$` (letters, numbers, dots, underscores, hyphens only) Target Virtual Cluster for this alias Pattern: `^[a-zA-Z0-9._-]+$` (letters, numbers, dots, underscores, hyphens only) The actual physical topic name on Kafka Pattern: `^[a-zA-Z0-9._-]+$` (letters, numbers, dots, underscores, hyphens only) Can be any valid topic name, no prefix requirement Physical Kafka cluster hosting the topic * For **Partner** Virtual Clusters: required, must be explicitly specified * For **Standard** Virtual Clusters: optional, defaults to `main` (only valid value) * Must match one of the Gateway's configured physical clusters ## TopicView A topic view filters records from a physical Kafka topic using SQL WHERE clauses, presenting a filtered read-only view to consumers. Topic views are useful when you need to expose a subset of data based on field values. Topic views will replace SQL topic Interceptors once they support the remaining SQL topic functionality. * **Managed with:** API, CLI ```yaml theme={null} --- apiVersion: gateway/v2 kind: TopicView metadata: name: uk-customers vCluster: my-vcluster spec: transformation: type: sql statement: "SELECT * FROM customers WHERE country='UK'" ``` **TopicView fields:** The logical name for the topic view that consumers will use Pattern: `^[a-zA-Z0-9._-]+$` (letters, numbers, dots, underscores, hyphens only) Target Virtual Cluster for this topic view Pattern: `^[a-zA-Z0-9._-]+$` (letters, numbers, dots, underscores, hyphens only) The transformation type. Currently only `sql` is supported. CEL (Common Expression Language) filtering will be available as an alternative in a future release. SQL SELECT statement defining the view * The `FROM` clause specifies the physical topic to read from (must be the full physical topic name) * The `WHERE` clause filters which records are returned * Use `SELECT *` to return all fields from matching records Example: `SELECT * FROM orders WHERE region='EU' AND status='pending'` [Find out more about topic views](/guide/conduktor-concepts/logical-topics#topic-views). ## Related resources * [Use and configure Interceptors](/guide/conduktor-concepts/interceptors) * [View Interceptor reference](/guide/reference/interceptor-reference) * [Give us feedback/request a feature](https://conduktor.io/roadmap)