Service Accounts, Authentication & Authorization
Introduction
This page explains how the Conduktor Gateway manages Service Accounts, client authentication, and ACLs (Access Control Lists).
A Service Account is a non-human identity used by clients to authenticate and perform actions on Kafka resources through Gateway, depending on their ACLs.
This page will help you understand:
- Where to authenticate clients, Gateway or Kafka
- Which authentication methods to use
- When you would want to use local, or external, Service Accounts
- How and where to manage service account ACLs
Local and External Service Accounts
In Gateway, you can define two types of Service Accounts:
- Local Service Account: An identity created and managed using the Gateway admin API, with no external dependency. Gateway provides a means to generate a Service Account password with a configurable time-to-live. These credentials can then be shared with clients for authentication, as such this is only supported in non-delegated mode. This type of Service Account is useful when you don't want to depend on an external identity provider. For example, when sharing data with an external partner and you don't want to manage a separate identity provider.
- External Service Account: An identity managed externally, such as by an OIDC identity provider, or an mTLS certificate. Optionally, this identity can be given a more user-friendly name in the Gateway. For instance, if Azure is used as the OIDC provider, the application principal recognized by Gateway defaults to a UUID generated by Azure. To enhance readability, this principal can be mapped to a Gateway external Service Account, allowing the use of a friendly name when declaring ACLs, Interceptors, and Audit Logs within the Gateway.
Each Service Account is stored in an internal topic, _conduktor_gateway_usermappings, and includes a name (used when applying ACLs and interceptors) and an associated Virtual Cluster. By default, this is set to passthrough.
Not all authentication methods support both local and external Service Accounts.
Please refer to the Client Authentication Methods section to see which method supports which type of Service Account.
Local Service Accounts
The local Service Accounts are useful if you want to manage the clients credentials directly within Conduktor Gateway. You can easily create, update, and delete them directly from Gateway's Admin API.
Learn how to manage a local service account.
External Service Accounts
For external Service Accounts, the clients credentials are created and handled by a third-party identity provider (OIDC, mTLS).
However, you might want to:
- Refer to them using a friendly name in Gateway
- Associate them to a Virtual Cluster
In these scenarios, you should create an external Service Account in Gateway, and link it to the principal given by your identity provider.
This external Service Account will be the one used in Gateway to apply ACLs & interceptors, and will be logged in the Gateway Audit Log internal topic.
Learn how to manage an external service account here.
Client Authentication Methods
As mentioned earlier, Gateway supports different clients authentication methods, but not all of them support local and external service accounts.
You'll find more details below, but here is a summary of the supported service accounts for each client's authentication method:
GATEWAY_SECURITY_PROTOCOL | Local SA | External SA |
|---|---|---|
| Anonymous | ||
| PLAINTEXT | ❌ | ❌ |
| SSL | ❌ | ❌ |
| SSL with client auth (mTLS) | ||
| SSL | ❌ | ✅ |
| SASL | ||
| SASL_PLAINTEXT | ✅ | only if OAUTHBEARER |
| SASL_SSL | ✅ | only if OAUTHBEARER |
| Delegated SASL | ||
| DELEGATED_SASL_PLAINTEXT | ❌ | ✅ |
| DELEGATED_SASL_SSL | ❌ | ✅ |
Anonymous Authentication
PLAINTEXT
In the case of PLAINTEXT authentication, the client is anonymous and doesn't need any credentials.
Thus, authentication cannot take place and so local and external Service Accounts are not supported in this mode.
How to configure the client > Gateway connection with PLAINTEXT.
SSL (Encryption Only)
If you use SSL for encryption only, Gateway presents a keystore certificate trusted by the client’s truststore, without authenticating the client.
Thus, local and external Service Accounts are not supported in this mode.
How to configure the client > Gateway connection with SSL.
SSL with Client Authentication (mTLS)
With mutual TLS (mTLS) authentication, both Kafka clients and Gateway validate each other’s identities using TLS certificates, ensuring secure and trusted bidirectional communication. This means that both the client and Gateway authenticate one another through their respective certificates.
As a result, Gateway extracts the user identity from the TLS certificate, which can be mapped to an external Service Account in Gateway.
The username will be of the form CN=writeuser,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown.
You can change that by setting the GATEWAY_SSL_PRINCIPAL_MAPPING_RULES environment variable to a customized rule. By default, it extracts the certificate distinguished name.
For instance: GATEWAY_SSL_PRINCIPAL_MAPPING_RULES=RULE:^CN=([a-zA-Z0-9.-]*).*$$/$$1/ , DEFAULT will extract the CN part of the certificate.
How to configure the client > Gateway connection with SSL mTLS.
SASL Authentication
With SASL authentication using OAUTHBEARER, clients authenticate with an identity (the sub in the OIDC JWT token) which can be mapped to an external Service Account in Gateway.
If you have configured OAUTHBEARER, Gateway expects the client to provide a JWT token, and the grant type should be clientcredentials.
See how to manage Gateway Service Accounts using SASL_PLAINTEXT.
It is the same for both SASL_PLAINTEXT and SASL_SSL. The only difference is that SASL_SSL encrypts the communication, while SASL_PLAINTEXT transmits in plain text.
How to configure the client > Gateway connection with SASL.
With Conduktor Gateway, you can decide where you'd like the client authentication & authorization to be made. This means that you can either:
- Delegate them at the backing Kafka cluster (Delegated SASL Authentication) - Gateway forwards the clients credentials to the Kafka cluster to authenticate them and retrieve their ACLs
- Handle them at Gateway (using the supported Clients Authentication Methods) - Gateway authenticates the clients and manages their ACLs
Delegated SASL Authentication
In some cases, you might want to delegate the authentication to the backing Kafka cluster. For example if you want to gradually transition to using the Conduktor Gateway, but first want to continue using the ACLs and Service Accounts defined in your Kafka cluster.
In this case, Gateway can forward the client credentials to the backing Kafka cluster, and the Kafka cluster will authenticate the client for Gateway.
Today, delegated SASL authentication supports only PLAIN and SCRAM mechanisms, not GSSAPI (Kerberos) or OAUTHBEARER. If you'd like us to support more mechanisms, please let us know!
In this mode, the clients authorization (ACLs) is also handled by the backing Kafka cluster. Any calls to the Kafka ACLs API made on Gateway will be forwarded to the backing Kafka cluster.
As such:
- Local Service Accounts are not available on Gateway
- External Service Accounts are available, and can be mapped to the client principal. This way Gateway will apply its interceptors on this external Service Account with a "friendly" name
Virtual resources (Virtual Clusters, Alias Topics and Concentrated Topics) are not available in the Delegated SASL mode.
How to configure the client > Gateway connection with DELEGATED_SASL.
Authorization Management (ACLs)
In Delegated Mode
Authorization (ACLs) is handled by the backing Kafka cluster.
Any calls to the Kafka ACLs API made on Gateway will be forwarded to the backing Kafka cluster.
In Non-Delegated Mode
Authorization (ACLs) is managed by Gateway, as such you have to define the ACLs for your applications in Gateway. This is done using the Kafka ACLs commands, or the Conduktor Console UI.
The principal attached to the ACLs can be either the local Service Account name, or the external Service Account name. If the client connects to Gateway using OAUTHBEARER, but no external Service Account is defined, the sub of the JWT client token will be used as the principal.