> ## Documentation Index > Fetch the complete documentation index at: https://docs.conduktor.io/llms.txt > Use this file to discover all available pages before exploring further. # Self-service resources > YAML resource definitions for Conduktor Self-service: Applications, ApplicationInstances, and permission schemas for GitOps-driven Kafka governance. ## Application An application represents a streaming app or data pipeline that's responsible for producing, consuming or processing data in Kafka. In Self-service, it's used as a method to organize and re-group multiple deployments of the same application (dev, prod) or different microservices that belong to the same team under one umbrella. * **API key(s):** AdminToken * **Managed with:** API, CLI, TF * **Labels support:** Full ```yaml theme={null} # Application --- apiVersion: self-serve/v1 kind: Application metadata: name: "clickstream-app" spec: title: "Clickstream App" description: "FreeForm text, probably multiline markdown" owner: "groupA" # technical-id of the Conduktor Console Group policyRef: - "applicationgroup-restrictions" ``` **Application checks:** * `spec.owner` is a valid Console group * `spec.policyRef` (optional), if set, has to be a valid list of [ResourcePolicy](#resourcepolicy). * Delete **has to fail** if there are associated `ApplicationInstance` [View Terraform documentation](https://registry.terraform.io/providers/conduktor/conduktor/latest/docs/resources/console_application_v1) **Side effects:** * None, deploying this object will only create the application in Console that can be managed on the **Application Catalog** page. ## ApplicationInstance Application instance represents an actual deployment of an application on a Kafka cluster for a service account. This is the core concept of Self-service, as it **ties everything together**: Kafka cluster, service account, ownership of resources and policies. * **API key(s):** AdminToken * **Managed with:** API, CLI, TF * **Labels support:** Full ```yaml theme={null} --- apiVersion: self-serve/v1 kind: ApplicationInstance metadata: application: "clickstream-app" name: "clickstream-dev" spec: cluster: "shadow-it" serviceAccount: "sa-clicko" policyRef: - "generic-dev-topic" - "clickstream-naming-rule" defaultCatalogVisibility: PUBLIC # makes all owned topics visible in the Topic Catalog by default resources: - type: TOPIC patternType: PREFIXED name: "click." - type: CONSUMER_GROUP patternType: PREFIXED name: "click." - type: SUBJECT patternType: PREFIXED name: "click." - type: TRANSACTIONAL_ID patternType: LITERAL name: "clickstream-txn-id" - type: CONNECTOR connectCluster: shadow-connect patternType: PREFIXED name: "click." - type: TOPIC patternType: PREFIXED ownershipMode: LIMITED # Topics are still maintained by central team name: "legacy-click." ``` **AppInstance checks:** * `metadata.application` is a valid application. * `spec.cluster` is a valid Console cluster technical Id. * `spec.cluster` is immutable (can't be updated after creation). * `spec.serviceAccount` (optional). If already used by another *AppInstance* on the the same `spec.cluster`, it can't be set. * `spec.applicationManagedServiceAccount` (optional), default is `false`. If set to `true`, the service account ACLs will be managed by the application owners directly instead of being synchronized by the *ApplicationInstance*. [Find out more about managed service account](#application-managed-service-account). * `spec.policyRef` (optional), if set, has to be a valid list of [ResourcePolicy](#resourcepolicy). * `spec.topicPolicyRef` (optional), if defined, has to be a valid list of [TopicPolicy](#topicpolicy). Will be deprecated in a future release — prefer `spec.policyRef` with [ResourcePolicy](#resourcepolicy) instead. * `spec.defaultCatalogVisibility` (optional), default is `PUBLIC`. Can be `PUBLIC` or `PRIVATE`. * `spec.resources[].type` can be `TOPIC`, `CONSUMER_GROUP`, `SUBJECT`, `TRANSACTIONAL_ID` or `CONNECTOR`: * `spec.resources[].connectCluster` is **only mandatory** when `type` is `CONNECTOR`; * `spec.resources[].connectCluster` is a valid Connect cluster linked to the Kafka cluster `spec.cluster`. * `spec.resources[].patternType` can be `PREFIXED` or `LITERAL`. * `spec.resources[].name` has to not overlap with any other *ApplicationInstance* on the same cluster. I.e.: if there's already an owner for `click`, this is forbidden: * `click.orders.`: resource is a child-resource of `click` * `cli`: resource is a parent-resource of `click` * `spec.resources[].ownershipMode` (optional), default is `ALL`. Can be `ALL` or `LIMITED`. [View Terraform documentation](https://registry.terraform.io/providers/conduktor/conduktor/latest/docs/resources/console_application_instance_v1) **Side effects:** * Console * Members of the owner group can create ApplicationGroups. * To create API keys, request or grant access, or manage service accounts, assign the corresponding `instancePermissions` to an [ApplicationGroup](#applicationgroup). * Resources with `ownershipMode` set to `ALL`: *ApplicationInstance* is given **all permissions** in the UI and the CLI over the owned resources. * Resources with `ownershipMode` set to `LIMITED`: *ApplicationInstance* is restricted the **create/update/delete permissions** in the UI and the CLI over the owned resources: * can't use the CLI `apply` command * can't create/delete the resource in the UI * everything else (restart connector, browse and produce from topic, etc.) is still available. [Find out more about ownership](/guide/use-cases/self-service#limited-ownership-mode). * Kafka * Service account is granted the following ACLs over the declared resources depending on the type: * Topic: `READ`, `WRITE` and `DESCRIBE_CONFIGS` * ConsumerGroup: `READ` * TransactionalId: `WRITE` and `DESCRIBE` * For Confluent Cloud and Confluent Platform clusters with their provider settings set to have role bindings enabled, the following RBAC role bindings will be created for the service account instead: * Topic: `DeveloperRead`, `DeveloperWrite` * There's also an implicit permission granted for subjects that share the same topic prefix. If there's `write` access to a topic, the service account will also receive `write` access over the subject. * Subject: `DeveloperRead`, `DeveloperWrite` * ConsumerGroup: `DeveloperRead` * TransactionalId: `DeveloperRead`, `DeveloperWrite` Find out how to migrate RBAC role bindings to [Confluent Cloud](/guide/tutorials/migrate-confluent-cloud-rbac) or [Confluent Platform](/guide/tutorials/migrate-confluent-platform-rbac). ### ApplicationInstancePermission Define permissions for the application instance to enable collaboration between teams. * **API key(s):** AdminToken, AppToken * **Managed with:** API, CLI, TF * **Labels support:** Missing ```yaml theme={null} # Permission granted to other applications --- apiVersion: self-serve/v1 kind: ApplicationInstancePermission metadata: application: "clickstream-app" appInstance: "clickstream-app-dev" name: "clickstream-app-dev-to-another" spec: resource: type: TOPIC name: "click.event-stream.avro" patternType: LITERAL userPermission: NONE serviceAccountPermission: READ grantedTo: "another-appinstance-dev" ``` **Application instance permission checks:** * `spec` is immutable: * once created, you'll only be able to update its metadata. **This is to protect you from making a change that could impact an external application**. * this resource affects target *ApplicationInstance*'s Kafka service account ACLs. * to edit this resource, delete and re-create it. * `spec.resource.type` can be `TOPIC`. * `spec.resource.patternType` can be `PREFIXED` or `LITERAL`. * `spec.resource.name` has to reference any 'sub-resource' of `metadata.appInstance`. For example, if you're the owner of the `click.` prefix, you can grant `READ` or `WRITE` access to: * the whole `click.` prefix, * a sub prefix `click.orders.`, * a literal topic name `click.orders.france`. * `spec.userPermission` can be `READ`, `WRITE` or `NONE`. * `spec.serviceAccountPermission` can be `READ`, `WRITE` or `NONE`. * `spec.userPermission` can be `READ` or `WRITE`. * `spec.serviceAccountPermission` can be `READ` or `WRITE`. * `spec.grantedTo` has to be an *ApplicationInstance* on the same Kafka cluster as `metadata.appInstance`. [View Terraform documentation](https://registry.terraform.io/providers/conduktor/conduktor/latest/docs/resources/console_application_instance_permission_v1) **Side effects:** * Console * Members of the `grantedTo` *ApplicationInstance* are given the associated permissions (`Read`/`Write`) in the UI over the resources. * Kafka * Service account of the `grantedTo` *ApplicationInstance* is granted to the following ACLs over the `resource`, depending on the `spec.permission`: * `READ`: READ, DESCRIBE\_CONFIGS * `WRITE`: READ, WRITE, DESCRIBE\_CONFIGS * For Confluent Cloud and Confluent Platform clusters with their provider settings set to have role bindings enabled, the following RBAC role bindings will be created for the service account instead: * `READ`: `DeveloperRead` * `WRITE`: `DeveloperRead`, `DeveloperWrite` * There's also an implicit permission granted for subjects when a topic permission is given. If there's `write` access to a topic called **example**, the service account will also receive `write` access to the subject **example**. Find out how to migrate RBAC role bindings to [Confluent Cloud](/guide/tutorials/migrate-confluent-cloud-rbac) or [Confluent Platform](/guide/tutorials/migrate-confluent-platform-rbac). ## ApplicationGroup Creates an application group to directly reflect how your application operates. You can create as many application groups as required - to restrict or enable the different teams that use Console. For example: * the support team can only have `Read` access in production environment, * the devOps team has extended access across all environments, * the engineering team is granted higher permissions in dev environment only. * **API key(s):** "AdminToken", "AppToken" * **Managed with:** API, CLI, UI, TF * **Labels support:** "MissingLabelSupport" #### Example ```yaml theme={null} # Permissions granted to Console users in the application --- apiVersion: self-serve/v1 kind: ApplicationGroup metadata: application: "clickstream-app" name: "clickstream-support" spec: displayName: Support Clickstream description: | Members of the Support Group are allowed: Read access on all the resources Can restart owned connectors Can reset offsets Can request and grant access instancePermissions: - appInstance: clickstream-app-dev permissions: ["applicationInstancePermissionRequestAccess", "applicationInstancePermissionGrantAccess"] permissions: - appInstance: clickstream-app-dev resourceType: TOPIC patternType: "LITERAL" name: "*" # All owned and subscribed topics permissions: ["topicViewConfig", "topicConsume"] - appInstance: clickstream-app-dev resourceType: CONSUMER_GROUP patternType: "LITERAL" name: "*" # All owned consumer groups permissions: ["consumerGroupCreate", "consumerGroupReset", "consumerGroupDelete", "consumerGroupView"] - appInstance: clickstream-app-dev connectCluster: local-connect resourceType: CONNECTOR patternType: "LITERAL" name: "*" # All owned connectors permissions: ["kafkaConnectViewConfig", "kafkaConnectStatus", "kafkaConnectRestart"] members: - user1@company.org - user2@company.org externalGroups: - GP-COMPANY-CLICKSTREAM-SUPPORT externalGroupRegex: - GP-COMPANY* ``` **ApplicationGroup checks:** * `spec.instancePermissions` (optional) is a list of instance-level permission assignments. * `spec.instancePermissions[].appInstance` has to be an application instance associated with this application (`metadata.application`). * `spec.instancePermissions[].permissions` is a list of valid instance permissions: `applicationInstancePermissionRequestAccess`, `applicationInstancePermissionGrantAccess`, `applicationInstancePermissionProposeAccess`, `applicationInstanceApiKeyManage`, `serviceAccountManage`. * `spec.permissions[].appInstance` has to be an application instance associated with this application (`metadata.application`). * `spec.permissions[].resourceType` can be `TOPIC`, `SUBJECT`, `CONSUMER_GROUP` or `CONNECTOR`. When set to `CONNECTOR`, an additional field `spec.permissions[].connectCluster` is mandatory and has to be a valid *KafkaConnectCluster* name. * `spec.permissions[].patternType` can be `PREFIXED` or `LITERAL`. * `spec.permissions[].name` has to reference any 'sub-resource' of `metadata.appInstance` or any subscribed topic. Use `*` to include to all owned and subscribed resources associated to this *appInstance*. * `spec.permissions[].permissions` are valid permissions. * `spec.members` has to be an email addresses of members that you want to add to this group. * `spec.externalGroups` a list of LDAP or OIDC groups to sync with this Console group. Members added this way will not appear in `spec.members` list. * `spec.externalGroupRegex` a list of regex patterns that can match to a series of LDAP or OIDC groups to sync with this Console group. Members added this way will not appear in `spec.members` list. [View Terraform documentation](https://registry.terraform.io/providers/conduktor/conduktor/latest/docs/resources/console_application_group_v1) **Side effects:** * Console * Members of the *ApplicationGroup* are given the associated resource permissions (`spec.permissions`) in the UI over the resources. * Members with `instancePermissions` can perform the corresponding actions on the application instance: * `applicationInstancePermissionRequestAccess`: request access to other teams' topics through the Topic Catalog. * `applicationInstancePermissionGrantAccess`: approve or grant access requests to owned topics. * `applicationInstancePermissionProposeAccess`: propose access via GitOps — view incoming access requests and copy the CLI snippet, without being able to approve them directly from the UI. * `applicationInstanceApiKeyManage`: create and delete application instance API keys. * `serviceAccountManage`: manage service accounts for the application instance. * Members of the LDAP or OIDC groups will be automatically added or removed upon login. ### Instance permissions reference Instance permissions control application-instance-level actions, separate from resource-level permissions. | Permission | Description | | -------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------- | | `applicationInstancePermissionRequestAccess` | Request read or write access to topics owned by other applications through the Topic Catalog. | | `applicationInstancePermissionGrantAccess` | Approve or grant incoming access requests to topics owned by this application instance. | | `applicationInstancePermissionProposeAccess` | Propose access via GitOps — view incoming access requests and copy the CLI snippet, without being able to approve them directly from the UI. | | `applicationInstanceApiKeyManage` | Create and delete API keys for this application instance. | | `serviceAccountManage` | Manage service accounts for this application instance (requires [application-managed service account](#application-managed-service-account) to be enabled). | ## Application-managed service account The Self-service service account is not configured by the central team at the `ApplicationInstance` level. Instead, the central platform team decides to delegate this responsibility to the application team, which needs to declare their own service account(s) and associated ACLs within the limits of what the `ApplicationInstance` is allowed to do. * **API key(s):** AppToken * **Managed with:** API, CLI * **Labels support:** Missing ```yaml theme={null} --- apiVersion: v1 kind: ServiceAccount metadata: appInstance: "clickstream-app-dev" cluster: shadow-it name: clickstream-sa spec: authorization: type: KAFKA_ACL acls: - type: TOPIC name: click.event-stream.avro patternType: PREFIXED operations: - Write - Read - type: CLUSTER name: kafka-cluster patternType: LITERAL operations: - DescribeConfigs - type: CONSUMER_GROUP name: cg-name patternType: LITERAL operations: - Read - type: TRANSACTIONAL_ID name: clickstream-txn-id patternType: LITERAL operations: - Write - Describe ``` **Service account checks:** The checks are the same as the [service account](/guide/reference/kafka-reference/#service-account) resource with additional **limitations**: * a service account is claimed by the first application team declaring it. * ACL operations that are not aligned with Self-service approach or would prevent configured policies to apply, are not allowed on service account: * **Topic**: topic name has to refer to a topic owned by *ApplicationInstance* or allowed by granted *ApplicationInstancePermission*: `Describe`, `DescribeConfigs`, `Read`, `Write`. * **Consumer group**: resource name has to refer to a consumer group owned by *ApplicationInstance* with `Describe` and `Read`. * **Cluster**: `Describe` and `DescribeConfigs`. * **Transactional Id**: resource name has to refer to a transactional ID owned by *ApplicationInstance* with `Write` and `Describe`. * **Delegation token**: out of scope, has to be assigned by a central team. * When an *ApplicationInstancePermission* is removed, we don't drop the ACLs on the *ServiceAccount*. Instead, consecutive CLI calls to apply the resource will fail. ## ResourcePolicy Resource policies enforce rules on resources across your Kafka infrastructure. They can be linked at the [KafkaCluster](/guide/reference/console-reference#kafkacluster), [KafkaConnectCluster](/guide/reference/console-reference#kafkaconnectcluster), [Application](#application) or [ApplicationInstance](#applicationinstance) level. Typical use cases include: * enforcing [Topic](/guide/reference/kafka-reference#topic) partition counts, replication factor and min in-sync replicas * enforcing [Topic](/guide/reference/kafka-reference#topic) naming conventions * enforcing business metadata conventions using [resource labels](/guide/use-cases/self-service#resource-labels) * restricting allowed [Connector](/guide/reference/kafka-reference#connector) plugin classes or capping `tasks.max` * enforcing [Subject](/guide/reference/kafka-reference#subject) compatibility levels or naming conventions * restricting what permissions can be given in [ApplicationGroups](#applicationgroup) created by an Application Resource policies are not applied automatically. You have to explicitly link them to an [ApplicationInstance](#applicationinstance) or [Application](#application) with `spec.policyRef`, or to a [KafkaCluster](/guide/reference/console-reference#kafkacluster) or [KafkaConnectCluster](/guide/reference/console-reference#kafkaconnectcluster) with `spec.policiesRef`. * **API key(s):** AdminToken * **Managed with:** API, CLI, TF * **Labels support:** Partial ```yaml theme={null} --- apiVersion: self-serve/v1 kind: ResourcePolicy metadata: name: "generic-dev-topic" labels: business-unit: delivery spec: targetKind: Topic description: A policy to check some basic rule for a topic rules: - condition: spec.replicationFactor == 3 errorMessage: replication factor should be 3 - condition: int(string(spec.configs["retention.ms"])) >= 60000 && int(string(spec.configs["retention.ms"])) <= 3600000 errorMessage: retention should be between 1m and 1h --- apiVersion: self-serve/v1 kind: ResourcePolicy metadata: name: "clickstream-naming-rule" labels: business-unit: delivery spec: targetKind: Topic description: A policy to check some basic rule for a topic rules: - condition: metadata.name.matches("^click\\.[a-z0-9-]+\\.(avro|json)$") errorMessage: topic name should match ^click\.(?[a-z0-9-]+)\.(avro|json)$ - condition: metadata.labels["data-criticality"] in ["C0", "C1", "C2"] errorMessage: data-criticality should be one of C0, C1, C2 ``` **SelfServicePolicy checks:** * `spec.targetKind` can be `Topic`, `Connector`, `Subject` or `ApplicationGroup`. * `spec.rules[].condition` is a valid [CEL expression](https://cel.dev) and will be evaluated against the resource. * `spec.rules[].errorMessage` is a string that will be displayed when the condition is not met. [Use this CEL playground](https://playcel.undistro.io/) to test your expressions. With the two policies declared, the following topic resource will succeed validation: ```yaml theme={null} --- apiVersion: kafka/v2 kind: Topic metadata: cluster: shadow-it name: click.event-stream.avro # Checked by metadata.name.matches("^click\\.[a-z0-9-]+\\.(avro|json)$") labels: data-criticality: C2 # Checked by metadata.labels["data-criticality"] in ["C0", "C1", "C2"] spec: replicationFactor: 3 # Check by spec.replicationFactor == 3 partitions: 3 configs: cleanup.policy: delete retention.ms: '60000' # Check int(string(spec.configs["retention.ms"])) >= 60000 && int(string(spec.configs["retention.ms"])) <= 3600000 ``` [View Terraform documentation](https://registry.terraform.io/providers/conduktor/conduktor/latest/docs/resources/console_resource_policy_v1) ### Restricting ApplicationGroup permissions After the [Console 1.45.0 owner group migration](/guide/release-notes#self-service-owner-group-permissions-v1-45-0), application teams manage their own ApplicationGroups and decide which `permissions` and `instancePermissions` to assign. Platform teams can use a `ResourcePolicy` with `targetKind: ApplicationGroup` to enforce guardrails on what teams are allowed to grant themselves. A common pattern is **GitOps-only approval**: the platform team mandates that approving access requests and any write operation on resources has to go through the CLI (typically from a peer-reviewed pull request), not directly in the Console UI. The policy below restricts ApplicationGroups to a read-only resource permission set (`topicViewConfig`, `consumerGroupView`, `subjectView`, `kafkaConnectorStatus`) and to three safe instance permissions: `applicationInstancePermissionRequestAccess`, `applicationInstancePermissionProposeAccess` and `applicationInstanceApiKeyManage`. Permissions like `topicDelete`, `topicEditConfig`, `applicationInstancePermissionGrantAccess` or `serviceAccountManage` are not allowed, so any approval, edit or delete has to happen through GitOps. ```yaml theme={null} --- apiVersion: self-serve/v1 kind: ResourcePolicy metadata: name: applicationgroup-gitops-only labels: business-unit: delivery spec: targetKind: ApplicationGroup description: Restrict ApplicationGroups to read-only resource permissions and GitOps-only access approval rules: - condition: spec.permissions.all(p, p.permissions.all(x, x in ["topicViewConfig", "consumerGroupView", "subjectView", "kafkaConnectorStatus"])) errorMessage: resource permissions are limited to topicViewConfig, consumerGroupView, subjectView and kafkaConnectorStatus. Permissions like topicDelete, topicEditConfig, consumerGroupReset and kafkaConnectorEditConfig have to go through the CLI - condition: '!has(spec.instancePermissions) || spec.instancePermissions.all(p, p.permissions.all(x, x in ["applicationInstancePermissionRequestAccess", "applicationInstancePermissionProposeAccess", "applicationInstanceApiKeyManage"]))' errorMessage: applicationInstancePermissionGrantAccess and serviceAccountManage are not allowed. Use applicationInstancePermissionProposeAccess and approve access via GitOps - condition: size(metadata.members) == 0 errorMessage: spec.members not allowed. Use external group mapping instead ``` With this policy linked through `spec.policyRef`, an ApplicationGroup like the following will pass validation — members can list and view resources, request and propose access, and manage API keys, but cannot approve incoming requests or perform any edit or delete from the UI: ```yaml theme={null} --- apiVersion: self-service/v1 kind: ApplicationGroup metadata: application: clickstream-app name: clickstream-support spec: displayName: Support Clickstream description: Read-only access with GitOps-only approval instancePermissions: - appInstance: clickstream-app-dev permissions: ["applicationInstanceApiKeyManage", "applicationInstancePermissionProposeAccess", "applicationInstancePermissionRequestAccess"] permissions: - appInstance: clickstream-app-dev resourceType: TOPIC patternType: LITERAL name: "*" permissions: ["topicViewConfig"] - appInstance: clickstream-app-dev resourceType: CONSUMER_GROUP patternType: LITERAL name: test permissions: ["consumerGroupView"] - appInstance: clickstream-app-dev resourceType: SUBJECT patternType: LITERAL name: test2 permissions: ["subjectView"] - appInstance: clickstream-app-dev connectCluster: connector resourceType: CONNECTOR patternType: LITERAL name: acme-connector permissions: ["kafkaConnectorStatus"] externalGroups: - GP-COMPANY-CLICKSTREAM-SUPPORT ``` Adding any other permission — for example `topicDelete`, `topicEditConfig`, `applicationInstancePermissionGrantAccess` or `serviceAccountManage` — will be rejected when the ApplicationGroup is applied, directing the team to make those changes through their GitOps workflow instead. ### Moving from TopicPolicy To replicate the behavior of the *TopicPolicy* with the `ResourcePolicy`, here's how you can transform the different policies: #### Range constraint Before: ```yaml theme={null} spec.configs.retention.ms: constraint: Range max: 3600000 min: 60000 ``` After: [open in playground](https://playcel.undistro.io/?content=H4sIAAAAAAAAA4VTXWvcMBD8K1s%2FJDmIfSGBPpimEMoVWq5t6FcodR%2F25D2fOGklJNmXo%2FS%2Fd6UzpH0otV%2FEzs7s7Fj%2BWSkyVVstl%2FBARjlLkBykHcGr1RruDR6H4Ebun3UsLX%2FXQEdABs2JAqqkJxKNzV2MZDfmCN4dKFAPxJMOji1xytr06I0LJMw%2Bnynoghx02p3mOmsdw%2BrRB4pRy3GNPIw4EFzI%2FEWTnRQ3NTwEnQiObgzFGj1xdjJ6bvoSqQhjIASB8jnqnmDrgpj3Y4IeE17KGb7dvVuDlN9%2B%2BvA%2B4xbTrHKfleH848jnZYsJzYj%2FGI4Dao6pTHoaMAut5v0zTzljSJITjtuKAlpvKM7GotcBM9bJKyFfRE%2BqUY63eojfuypQkuAEb2zsqh8LeHkLz6%2FkgbMzKP0paB7%2BR1vAi1u4KcSr6rLKTt9kz3Ip6rruGL3%2BSiHv1cIet3tcTtcd7zX3LXx2XqtOPm3CzGs7BlBmjHIhWog77N2h1ilXGS21gmm1b2gSA7W4I7QNTsHlBoMbMrEoAPTOSoJz%2F6nxBKD3yvVFad9x3qwwAnnpLGG9lpvoZPpNrnsMSedqnAtzDPMYq7mRnI%2BsmllB%2BrrquqtOuDKEPPrGO8GOLfRkJLwT9meMmVQCzETJ0IpFiS%2F%2FWb9%2BA%2Bwuz6hhAwAA) ```yaml theme={null} - condition: int(string(spec.configs["retention.ms"])) >= 60000 && int(string(spec.configs["retention.ms"])) <= 3600000 errorMessage: retention should be between 1m and 1h ``` #### Value constraint Before: ```yaml theme={null} spec.replicationFactor: constraint: OneOf values: ["3"] ``` After: [open in playground](https://playcel.undistro.io/?content=H4sIAAAAAAAAA3VTTWvcMBD9K1Nf0kLsLQn0YMghlC20bNvQr1DwZVae9QpLIyHJ3iyl%2F70j2W0oIdZlmDfvvfnAvypFpmqrzQbuyShnCZKDdCR4u93BncHzENzE%2FYuOpeT%2FHOgIyKA5UUCV9Eyisb%2BNkezenMG7EwXqgXjWwbElTlmbHrxxgYTZ55iCLshJp%2BPi66x1DNsHHyhGLeEOeZhwIHgp%2Fq%2Ba3Enppob7oBPB2U2htEaPnKNYr0XfIxVhDIQgUI6j7gkOLkjzfkrQY8JLieHn7ccdSPrD18%2BfMm4xrSp3WRkuvkx8UaaY0Uz4jDkOqDmm4vRosApt1%2FkzTzljSDYnHHcQBbTeUFwbi14HzFgnL3pSTSBvtCq5d7Jwqbq5gevqssry77ORXLKu647R6x8UcjMtjHgYcTNfdTxq7lv45rxWndwjYea1HQMoM0W5YgvxiL071TrlLKOlVjCtxoZmuVIdkyzRNjgHlwsM7snEogDQOytjr%2FVL4QKg98r1RWlcJimMJ9O0cJ3zHkPSORvXhHJ80MNfG6u5keWc%2Bd8%2BpK6rrrpqwZUh5Mk33gl2bqEnQ4kWLEjEWbqxhfTmtXyZKDu00qKsL%2F8Ov%2F8A22by1BYDAAA%3D) ```yaml theme={null} - condition: spec.replicationFactor == 3 errorMessage: replication factor should be 3 ``` #### In list constraint Before: ```yaml theme={null} metadata.labels.data-criticality: constraint: OneOf values: ["C0", "C1", "C2"] ``` After: [open in playground](https://playcel.undistro.io/?content=H4sIAAAAAAAAA3VTTY%2FTQAz9KyaXBalJl0XikNtqVSRQgRUfu0KEgztxU6vzpZlJuhXiv%2BOZBFUgkRzi%2BNnPz2%2BSn5UiXbXVeg2PpJUzBMlBOhDcbbZwr%2FE8BDfa%2FllnpeTvHHAEtMA2UUCVeCLh2N3GSGanz%2BDdiQL1QHbi4KwhmzI3PXntAklnn2MKXJATp8M81xnjLGyefKAYWcIt2mHEgeC5zH%2FRZCVFTQ2PgRPB2Y2hSKNLz0FGL0VfIxViDIQgUI4j9wR7F0S8HxP0mHAlMXy7fb8FSb%2F7%2FPFDxg2mheU%2BM8PVp9FelS0m1CP%2BZzgOyDamMukyYCHaLPvnPuW0JnFOetxeGNB4TXERFj0HzFgnt6GEmaTRuCMdv3dVfquVGMAKNadzV%2F3IGwhyd91VK5Dny%2BV5I1i1Kh1vsxw577quO4ueHyhkyS0ccX%2FE9XTT2SPbvoUvzrO6zG07C6D0GOWsW4gH7N2p5pSzFg21grE6NjTJWdYxidWmwSm4XDBLLgwAvTNizlI%2FF84Aeq9cX5iOS%2Bk%2FG7Ygm4ktpApXIC8cxaE38vk50fUq5z2GxDkbl4Ryds%2FDHwGGbSPmnq1qFgap6yrxaMaVJrSjb7wTTGb2pCnRjAWJbKZuTGl6fS1XbhR3jYgXY%2FPv9Os311aOeVYDAAA%3D) ```yaml theme={null} - condition: metadata.labels["data-criticality"] in ["C0", "C1", "C2"] errorMessage: data-criticality should be one of C0, C1, C2 ``` #### Regex constraint Before: ```yaml theme={null} metadata.name: constraint: Match pattern: ^click\.(?[a-z0-9-]+)\.(avro|json)$ ``` After: [open in playground](https://playcel.undistro.io/?content=H4sIAAAAAAAAA3VTbYsTMRD%2BK%2BMiXIu32%2BMEwf12SAWl6uHbIa7CNDttY5NJSLLb66n%2F3Um6Uvzg7pfpPPO8ZLL9WSkyVVstFnBHRjlLkBykHcGL5QpuDR63wQ3cP%2BpYRv7tgY6ADJoTBVRJjyQa65sYya7NEbw7UKAeiEcdHFvilLXp3hsXSJh9rinoghx02p18nbWOYXnvA8WopVwhbwfcEszEf97kJCVNDXdBJ4KjG0KJRmfOTqynoU%2BRijAGQhAo11H3BBsXJLwfEvSY8FJq%2BHLzZgXSfv3h3duMW0yTym1Whov3A1%2BUU4xoBvyPOW5Rc0zF6WwwCS2n82eecsaQbE44biMKaL2hOAWLXgfMWCevpYRZpGG01EgstaM466rvymi17%2BRpvmL9cFU%2Fr789KT9nOAb360d0PH%2FcVfPqssr8VzmO3Hdd1x2j158p5Mgt7HGzx8V43fFec9%2FCR%2Be1Ovu2HQMoM0S56xbiDnt3qHXK3ZyohZKjoVHuso5JVm2bHCAPGFyTiUUBoHdWljPNnwZPAHqvXF%2BU9p0cnlRhBPIyWfbwUj4yJ%2B5Pc99jSDp349RQjjd6%2B9fGam5khUdWzaQgc1113VUnXBlCHnzjnWDHFnoylOiEBak4Sze2kJ5dyZOJskMrEWV9%2BU%2Fz%2Bw9HD%2BgYPAMAAA%3D%3D) ```yaml theme={null} - condition: metadata.name.matches("^click\\.[a-z0-9-]+\\.(avro|json)$") errorMessage: topic name should match ^click\.(?[a-z0-9-]+)\.(avro|json)$ ``` #### Tips for CEL expressions There are multiple things to consider when writing CEL expressions in the context of resource policies: * For field-like configuration value/label (that you don't know the type of) and want to compare to a number, convert it to a *string* and then to an *int* like this: `int(string(spec.configs["retention.ms"]))`. * For field key that contains dots `.` or dashes `-`, you have to access them with the `[]` operator: `metadata.labels["data-criticality"]`. * For field-like label key/config that can be absent, we recommend adding a check to see if the field is present: `has(metadata.labels.criticality) && {your condition}`. If the field has a dot or dash, use `"retention.ms" in spec.configs && {your condition}`. ## TopicPolicy Topic policies force application teams to conform to topic rules, set at their `ApplicationInstance` level. Typical use cases include: * safeguarding from invalid or risky topic configuration * enforcing a naming convention * enforcing metadata TopicPolicy will be deprecated in a future release. Please use [ResourcePolicy](#resourcepolicy) instead. * **API key(s):** AdminToken * **Managed with:** API, CLI, TF * **Labels support:** Partial ```yaml theme={null} --- apiVersion: self-serve/v1 kind: TopicPolicy metadata: name: "generic-dev-topic" spec: policies: metadata.labels.data-criticality: constraint: OneOf values: ["C0", "C1", "C2"] spec.configs.retention.ms: constraint: Range max: 3600000 min: 60000 spec.replicationFactor: constraint: OneOf values: ["3"] --- apiVersion: self-serve/v1 kind: TopicPolicy metadata: name: "clickstream-naming-rule" spec: policies: metadata.name: constraint: Match pattern: ^click\.(?[a-z0-9-]+)\.(avro|json)$ ``` **TopicPolicy checks:** * `spec.policies` require YAML paths that are paths to the [topic resource YAML](/guide/reference/kafka-reference#topic). For example: * `metadata.name` to create constraints on topic name * `metadata.labels.` to create constraints on topic label `` * `spec.partitions` to create constraints on partitions number * `spec.replicationFactor` to create constraints on replication factor * `spec.configs.` to create constraints on topic config `` * `spec.policies..constraint` can be `Range`, `OneOf` or `Match` With the two topic policies declared above, the following topic resource would succeed validation: ```yaml theme={null} --- apiVersion: kafka/v2 kind: Topic metadata: cluster: shadow-it name: click.event-stream.avro # Checked by Match ^click\.(?[a-z0-9-]+)\.(avro|json)$ on `metadata.name` labels: data-criticality: C2 # Checked by OneOf ["C0", "C1", "C2"] on `metadata.labels.data-criticality` spec: replicationFactor: 3 # Checked by OneOf ["3"] on `spec.replicationFactor` partitions: 3 configs: cleanup.policy: delete retention.ms: '60000' # Checked by Range(60000, 3600000) on `spec.configs.retention.ms` ``` [View Terraform documentation](https://registry.terraform.io/providers/conduktor/conduktor/latest/docs/resources/console_topic_policy_v1) ### Topic policy constraints There are currently five available constraints: * `Range` validates a range of numbers * `OneOf` validates against a list of predefined options * `NoneOf` rejects a value if it matches any item in the list * `Match` validates using a regex (regular expression) * `AllowedKeys` limits a set of keys in the dictionaries #### Range Validates whether the property belongs to a range of numbers (inclusive): ```yaml theme={null} spec.configs.retention.ms: constraint: "Range" min: 3600000 # 1 hour in ms max: 604800000 # 7 days in ms ``` Validation will succeed with these inputs: * 3600000 (min) * 36000000 (between min and max) * 604800000 (max) Validation will fail with these inputs: * 60000 (below min) * 999999999 (above max) #### OneOf Validates whether the property is one of the expected values: ```yaml theme={null} spec.configs.cleanup.policy: constraint: OneOf values: ["delete", "compact"] ``` Validation will succeed with these inputs: * `delete` * `compact` Validation will fail with these inputs: * `delete, compact` (valid in Kafka but not allowed by policy) * `deleet` (typo) #### Match Validates the property against a regex: ```yaml theme={null} metadata.name: constraint: Match pattern: ^wikipedia\.(?[a-z0-9]+)\.(avro|json)$ ``` Validation will succeed with these inputs: * `wikipedia.links.avro` * `wikipedia.products.json` Validation will fail with these inputs: * `notwikipedia.products.avro2`: `^` and `$` prevents anything before and after the pattern * `wikipedia.all-products.avro`: `(?[a-z0-9]+)` prevents anything else than lowercase letters and digits #### AllowedKeys Validates whether the keys are within an allowed key list. Applies to dictionary type (Key/Value maps). Can be used on `spec.configs` and `metadata.labels`. ```yaml theme={null} spec.configs: constraint: AllowedKeys keys: - retention.ms - cleanup.policy ``` Validation will succeed with this input: ```yaml theme={null} --- apiVersion: kafka/v2 kind: Topic metadata: cluster: shadow-it name: click.event-stream.avro spec: replicationFactor: 3 partitions: 3 configs: cleanup.policy: delete retention.ms: '60000' ``` Validation will fail with this input (`min.insync.replicas` is not an allowed key in `spec.configs`): ```yaml theme={null} --- apiVersion: kafka/v2 kind: Topic metadata: cluster: shadow-it name: click.event-stream.avro spec: replicationFactor: 3 partitions: 3 configs: min.insync.replicas: '2' # Not in AllowedKeys cleanup.policy: delete retention.ms: '60000' ``` #### Optional flag Constraints can be marked as optional. In this scenario, the constraint will only be validated if the field exists. E.g.: ```yaml theme={null} spec.configs.min.insync.replicas: constraint: ValidString optional: true values: ["2"] ``` This object will pass the validation: ```yaml theme={null} --- apiVersion: kafka/v2 kind: Topic metadata: cluster: shadow-it name: click.event-stream.avro spec: replicationFactor: 3 partitions: 3 configs: cleanup.policy: delete retention.ms: '60000' ``` This object will fail the validation due to a new incorrect definition of `insync.replicas`: ```yaml theme={null} --- apiVersion: kafka/v2 kind: Topic metadata: cluster: shadow-it name: click.event-stream.avro spec: replicationFactor: 3 partitions: 3 configs: min.insync.replicas: 3 cleanup.policy: delete retention.ms: '60000' ``` ## TopicTemplate Topic templates are admin-curated starting points for the Console **Create Topic** form. When an application team creates a topic, they can pick a template to pre-fill the partition count, replication factor, configs and labels, then adjust the values before they submit. Templates are suggestions, not rules — unlike a [ResourcePolicy](#resourcepolicy), they don't block anything. Pair the two to point teams at a sensible default and enforce the boundaries. Typical use cases include: * offering a high-throughput or compacted topic preset * suggesting a naming convention with `{{placeholder}}` values the user replaces * seeding the labels your governance model expects * **API key(s):** AdminToken * **Managed with:** API, CLI * **Labels support:** Partial ```yaml theme={null} --- apiVersion: v2 kind: TopicTemplate metadata: name: high-partition-topic labels: category: template spec: displayName: "High Partition Topic" description: "Optimized for high-throughput workloads. 24 partitions, 3x replication, 7-day retention." defaults: metadata: name: "my-topic-{{department}}" labels: throughput: high spec: partitions: 24 replicationFactor: 3 configs: cleanup.policy: delete retention.ms: "604800000" min.insync.replicas: "2" ``` **TopicTemplate checks:** * `metadata.name` is the unique identifier of the template. * `spec.displayName` is required and is the name shown in the **Create Topic** template picker. * `spec.description` (optional) appears under the display name to help users choose. * `spec.defaults` (all optional) pre-fill the form: `metadata.name`, `metadata.labels`, `spec.partitions`, `spec.replicationFactor` and `spec.configs`. * `spec.defaults.spec.partitions` and `spec.defaults.spec.replicationFactor`, when set, have to be greater than zero. * `{{placeholder}}` values are validated for syntax (`{{` followed by a letter or underscore, then letters, digits or underscores, then `}}`) but are never resolved on the server. The user replaces them in the form before they submit. **Side effects:** * The template appears in the **Create Topic** form for every user who can create topics. Selecting it pre-fills the form, and the user can still change any value. * Placeholders aren't substituted automatically. Console flags a topic name that still contains a `{{placeholder}}` and blocks submission until the user replaces it. ## ConnectorTemplate Connector templates work like [TopicTemplate](#topictemplate) for the Console connector wizard. They pre-fill the connector class, configuration and labels so application teams start from a vetted setup instead of a blank form. Like topic templates, they're suggestions rather than rules. Typical use cases include: * offering a standard sink preset, for example an S3 or Elasticsearch sink, with sensible batching and retry defaults * starting teams from an approved connector plugin class * suggesting a naming convention with `{{placeholder}}` values the user replaces * **API key(s):** AdminToken * **Managed with:** API, CLI * **Labels support:** Partial ```yaml theme={null} --- apiVersion: v2 kind: ConnectorTemplate metadata: name: s3-sink-standard labels: category: sink spec: displayName: "S3 Sink (Standard)" description: "Standard S3 sink connector with flush every 10,000 records or 60 seconds. Avro format on S3 storage." defaults: metadata: name: "s3-sink-{{department}}" labels: category: sink spec: class: io.confluent.connect.s3.S3SinkConnector config: tasks.max: "2" flush.size: "10000" rotate.interval.ms: "60000" storage.class: "io.confluent.connect.s3.storage.S3Storage" format.class: "io.confluent.connect.s3.format.avro.AvroFormat" ``` **ConnectorTemplate checks:** * `metadata.name` is the unique identifier of the template. * `spec.displayName` is required and is the name shown in the connector wizard template picker. * `spec.defaults.spec.class` is required — the connector plugin class the template starts from. The wizard offers the template only when the user selects a matching connector class. * `spec.defaults.spec.config` holds the connector configuration to pre-fill. Keys that look like secrets — for example `password`, `secret`, `credentials` or `sasl.jaas.config` — are rejected, so templates never store sensitive values. * `{{placeholder}}` values follow the same syntax and pass-through rules as [TopicTemplate](#topictemplate). **Side effects:** * The template appears in the connector wizard when a user selects the matching connector class. Selecting it pre-fills the configuration, and the user can still change any value. * As with topic templates, Console blocks submission while a `{{placeholder}}` is unresolved.