Service Accounts, authentication and authorization
Overview
Find out how Conduktor Gateway manages Service Accounts, client authentication and ACLs (Access Control Lists):
- where to authenticate clients: Gateway or Kafka
- which authentication method to use
- when to use local or external Service Accounts
- how and where to manage Service Account ACLs
A Service Account is a non-human identity, used by Kafka clients to authenticate and perform actions on resources through Gateway, depending on their ACLs.
Local and external Service Accounts
In Gateway, you can define two types of Service Accounts:
-
Local: an identity created and managed using the Gateway admin API with no external dependency. Gateway provides the means to generate a Service Account password with a configurable time-to-live. These credentials can then be shared with clients for authentication. as such this is only supported in non-delegated mode. This type of Service Account is useful when you don't want to depend on an external identity provider. For example, when sharing data with an external partner and you don't want to manage a separate identity provider.
-
External: an identity managed externally, such as by an OIDC identity provider or an mTLS certificate. Optionally, this identity can be given a more user-friendly name in Gateway. For instance, if Azure is used as the OIDC provider, the application principal recognized by Gateway defaults to a UUID generated by Azure. To enhance readability, this principal can be mapped to a Gateway external Service Account, letting you use a friendly name when declaring ACLs, Interceptors and Audit Logs within the Gateway.
Each Service Account is stored in an internal topic, _conduktor_${GATEWAY_CLUSTER_ID}_usermappings, and includes a name (used when applying ACLs and interceptors) and an associated Virtual Cluster. By default, this is set to passthrough.
See the supported client authentication methods matrix for details.
Local Service Accounts
The local Service Accounts are useful if you want to manage the clients credentials directly within Conduktor Gateway. You can easily create, update, and delete them directly from Gateway's Admin API.
Learn how to manage a local service account.
External Service Accounts
For external Service Accounts, the clients credentials are created and handled by a third-party identity provider (OIDC, mTLS).
However, you might want to:
- Refer to them using a friendly name in Gateway
- Associate them to a Virtual Cluster
In these scenarios, you should create an external Service Account in Gateway, and link it to the principal given by your identity provider.
This external Service Account will be the one used in Gateway to apply ACLs & interceptors, and will be logged in the Gateway Audit Log internal topic.
Learn how to manage an external service account here.
Client authentication methods
Gateway supports different client authentication methods, depending on whether you're using local or external Service Accounts.
GATEWAY_SECURITY_PROTOCOL | Local SA | External SA |
|---|---|---|
| Anonymous | ||
| PLAINTEXT | ❌ | ❌ |
| SSL | ❌ | ❌ |
| SSL with client auth (mTLS) | ||
| SSL | ❌ | ✅ |
| SASL | ||
| SASL_PLAINTEXT | ✅ | only if OAUTHBEARER |
| SASL_SSL | ✅ | only if OAUTHBEARER |
| Delegated SASL | ||
| DELEGATED_SASL_PLAINTEXT | ❌ | ✅ |
| DELEGATED_SASL_SSL | ❌ | ✅ |
Anonymous authentication
PLAINTEXT
In the case of PLAINTEXT authentication, the client is anonymous and doesn't need any credentials.
Thus, authentication cannot take place and so local and external Service Accounts are not supported in this mode.
How to configure the client > Gateway connection with PLAINTEXT.
SSL (Encryption Only)
If you use SSL for encryption only, Gateway presents a keystore certificate trusted by the client’s truststore, without authenticating the client.
Thus, local and external Service Accounts are not supported in this mode.
How to configure the client > Gateway connection with SSL.
SSL with Client Authentication (mTLS)
With mutual TLS (mTLS) authentication, both Kafka clients and Gateway validate each other’s identities using TLS certificates, ensuring secure and trusted bidirectional communication. This means that both the client and Gateway authenticate one another through their respective certificates.
As a result, Gateway extracts the user identity from the TLS certificate, which can be mapped to an external Service Account in Gateway.
The username will be of the form CN=writeuser,OU=Unknown,O=Unknown,L=Unknown,ST=Unknown,C=Unknown.
You can change that by setting the GATEWAY_SSL_PRINCIPAL_MAPPING_RULES environment variable to a customized rule. By default, it extracts the certificate distinguished name.
For instance: GATEWAY_SSL_PRINCIPAL_MAPPING_RULES=RULE:^CN=([a-zA-Z0-9.-]*).*$$/$$1/ , DEFAULT will extract the CN part of the certificate.
How to configure the client > Gateway connection with SSL mTLS.
SASL Authentication
With SASL authentication using OAUTHBEARER, clients authenticate with an identity (the sub in the OIDC JWT token) which can be mapped to an external Service Account in Gateway.
If you have configured OAUTHBEARER, Gateway expects the client to provide a JWT token, and the grant type should be clientcredentials.
See how to manage Gateway Service Accounts using SASL_PLAINTEXT.
It is the same for both SASL_PLAINTEXT and SASL_SSL. The only difference is that SASL_SSL encrypts the communication, while SASL_PLAINTEXT transmits in plain text.
How to configure the client > Gateway connection with SASL.
With Conduktor Gateway, you can decide where you'd like the client authentication & authorization to be made. This means that you can either:
- Delegate them to the backing Kafka cluster (Delegated SASL Authentication) - Gateway forwards the clients credentials to the Kafka cluster to authenticate them and retrieve their ACLs
- Handle them with Gateway (using the supported Clients Authentication Methods) - Gateway authenticates the clients and manages their ACLs
Delegated SASL Authentication
In some cases, you might want to delegate the authentication to the backing Kafka cluster. For example if you want to gradually transition to using the Conduktor Gateway, but first want to continue using the ACLs and Service Accounts defined in your Kafka cluster.
In this case, Gateway can forward the client credentials to the backing Kafka cluster, and the Kafka cluster will authenticate the client for Gateway.
Currently, delegated SASL authentication only supports PLAIN, SCRAM, OAUTHBEARER and AWS_MSK_IAM mechanisms. Get in touch for specific requirements.
In this mode, the clients authorization (ACLs) is also handled by the backing Kafka cluster. Any calls to the Kafka ACLs API made on Gateway will be forwarded to the backing Kafka cluster.
As such:
- Local Service Accounts are not available on Gateway
- External Service Accounts are available, and can be mapped to the client principal. This way Gateway will apply its interceptors on this external Service Account with a "friendly" name
Virtual resources (Virtual Clusters, Alias Topics and Concentrated Topics) are not available in the Delegated SASL mode.
How to configure the client > Gateway connection with DELEGATED_SASL.
Authorization Management (ACLs)
In Delegated Mode
Authorization (ACLs) is handled by the backing Kafka cluster.
Any calls to the Kafka ACLs API made on Gateway will be forwarded to the backing Kafka cluster.
In Non-Delegated Mode
Authorization (ACLs) is managed by Gateway, so you have to define the ACLs for your applications in Gateway. This is done using the Kafka ACLs commands, or the Conduktor Console UI.
The principal attached to the ACLs can either be the local Service Account name, or the external Service Account name. If the client connects to Gateway using OAUTHBEARER, but no external Service Account is defined, the sub of the JWT client token will be used as the principal.