Use the Gateway container
- Docker
- Kubernetes
You can set the environment variables during the docker-run command with Or in a
-e or --env:docker-compose.yaml:Use a file
You can mount a file with the key-value pairs into the container and provide its path by setting the environment variableGATEWAY_ENV_FILE. Note these variables should be exported by the file as they are injected in the wrapper that starts the Gateway process.
Example
Sourcing environment variables from $GATEWAY_ENV_FILE (or a warning if the file isn’t found: Warning: GATEWAY_ENV_FILE is set but the file does not exist or is not readable.).
Listeners
Per-listener configuration with
GATEWAY_LISTENER_<NAME>_* variables is available from Gateway 3.20.0. On earlier versions, networking is configured with the global variables described in Legacy network configuration. To move an existing deployment over, see Migrate to listener configuration.GATEWAY_LISTENER_<NAME>_<PROPERTY>, where <NAME> is a label you choose. See Configure Gateway listeners for details.
Listener names have to start with a letter and contain only alphanumeric characters. Underscores are reserved as delimiters in
GATEWAY_LISTENER_<NAME>_<PROPERTY>, so a name like MY_LISTENER would be ambiguous. Gateway only recognizes variables whose <NAME> matches this pattern — variables using any other name are ignored, and the listener is silently not created.
The following global variable applies to all listeners that use port routing:
Listeners: port specification format
GATEWAY_LISTENER_<NAME>_PORTS accepts ranges, individual values, or a combination of both, separated by commas:
advertised:local format:
GATEWAY_MIN_BROKERID=0 and three brokers:
advertised:local format maps the advertised port (for example, a load balancer’s 443) to the local one:
Listeners: SSL client authentication modes
TheGATEWAY_LISTENER_<NAME>_SSL_CLIENT_AUTH setting controls whether the Gateway requests a client certificate during the TLS handshake for a specific listener:
When
SSL_CLIENT_AUTH is set to OPTIONAL or REQUIRE, you must configure a truststore in sslConfig.trustStore containing the CA certificates used to verify client certificates.Listeners: zero-config mode
If you don’t set any listener or networking environment variables, Gateway configures itself automatically. It creates a single listener namedDEFAULT with port routing, assigning ports from 6969 (one per broker, plus two for headroom), and takes the listener’s security protocol from the backing Kafka cluster. GATEWAY_ACL_ENABLED defaults to false in this mode.
When GATEWAY_SECURITY_MODE is not set, Gateway infers it from the backing cluster’s security protocol:
Listeners: legacy configuration (deprecated)
These global variables are deprecated in favor of per-listener configuration. They remain supported for now. For how they behave, see Legacy network configuration; to switch, see Migrate to listener configuration.External connections (preview)
This is a preview feature and is subject to change. Declare connections to external systems using the patternGATEWAY_EXTERNAL_CONNECTION_<NAME>_CONFIG_<PROPERTY>, where <NAME> is a label you choose and is how a resource references the connection. See Configure Gateway external connections for details.
Connection names have to start with a letter and can contain letters, numbers and underscores, so
DEFAULT_SR is a valid name. A name can’t contain _CONFIG_, because that’s the delimiter between the name and the property. Gateway rejects such a variable at startup rather than guessing where the name ends. Names are case-insensitive and Gateway stores them in upper case, so default_sr and DEFAULT_SR are the same connection. We recommend you declare them in upper case to match the variable.TYPE:
External connections: Confluent-like schema registry
SetTYPE to CONFLUENT_SCHEMA_REGISTRY.
External connections: AWS Glue schema registry
SetTYPE to AWS_SCHEMA_REGISTRY.
Omit every
BASIC_CREDENTIALS_ variable and Gateway falls back to the AWS SDK’s default credentials provider. See what it searches.
External connections: schema registry client properties
Both schema registry types also accept any property their client library understands, in the same namespace. See the formats and precedence rules.External connections: HashiCorp Vault KMS
SetTYPE to HASHICORP_VAULT. These are the connection’s own properties. Its credentials depend on AUTH_TYPE and are listed in the three sections that follow.
Write
ENGINE, AUTH_TYPE and CONNECTION_BACKOFF_CHRONO_UNIT in upper case, as shown. Gateway matches these values exactly, so transit and seconds fail startup.
External connections: Vault token authentication
SetAUTH_TYPE to TOKEN.
External connections: Vault AppRole authentication
SetAUTH_TYPE to APP_ROLE.
External connections: Vault username and password authentication
SetAUTH_TYPE to USERNAME_PASSWORD.
External connections: AWS KMS
SetTYPE to AWS_KMS. These are the connection’s own properties. Its credentials depend on CREDENTIALS_TYPE and are listed in the three sections that follow.
Write
CREDENTIALS_TYPE in upper case, as shown. Gateway matches the value exactly, so basic fails startup.
External connections: AWS KMS access keys
SetCREDENTIALS_TYPE to BASIC.
External connections: AWS KMS temporary credentials
SetCREDENTIALS_TYPE to SESSION. Gateway doesn’t refresh these, so they stop working when they expire.
External connections: AWS KMS default provider chain
SetCREDENTIALS_TYPE to DEFAULT and pass no credentials. Gateway resolves them through the AWS SDK’s default provider chain. See what it searches.
External connections: startup checks
Applies to the connection types Gateway contacts at startup, which means the two schema registry types. Gateway doesn’t contact a KMS at startup.Load balancing
Connect Gateway to Kafka
Conduktor Gateway’s connection to Kafka is configured by theKAFKA_ environment variables.
When translating Kafka’s properties, use upper case instead and replace the . with _. For example:
When defining Gateway’s Kafka property bootstrap.servers, declare it as the environment variable KAFKA_BOOTSTRAP_SERVERS. Any variable prefixed with KAFKA_ will be treated as a connection parameter by Gateway.
Connection pooling
Cluster switch failover
Setting up your Kafka clusters for failover is similar to the standard setup, but you need to provide two sets of properties: one for your main cluster and one for your failover cluster. You can define these properties as environment variables or load a cluster configuration file.Security and authentication
These global settings determine where authentication and authorization take place, and how clients authenticate. The security protocol itself is set per listener — see Listeners.Set the security protocol clients use to connect to Gateway per listener, with
GATEWAY_LISTENER_<NAME>_SECURITY_PROTOCOL (see Listeners). The global GATEWAY_SECURITY_PROTOCOL and its deprecated DELEGATED_* values apply only to legacy network configuration.SSL authentication
The keystore and truststore are shared across all listeners. mTLS client authentication is set per listener with
GATEWAY_LISTENER_<NAME>_SSL_CLIENT_AUTH — the global GATEWAY_SSL_CLIENT_AUTH applies only to legacy network configuration.Password values (
GATEWAY_SSL_KEY_STORE_PASSWORD, GATEWAY_SSL_KEY_PASSWORD, GATEWAY_SSL_TRUST_STORE_PASSWORD) support any characters except a literal single-quote ('). Pattern: [^']*OAuthbearer
Some of these definitions (e.g.SASL_OAUTHBEARER_JWKS_ENDPOINT_REFRESH) are taken from Kafka documentation .
Principal resolver
Find out more about the principal resolver.
Plain authentication
Security provider
HTTP API
Password values (
GATEWAY_HTTPS_KEY_STORE_PASSWORD, GATEWAY_HTTPS_TRUST_STORE_PASSWORD) support any characters except a literal single-quote ('). Pattern: [^']*Feature flags
Topic ID support (preview)
Gateway supports topic-ID based Kafka protocol handling. Initial coverage is limited to the Fetch API: when disabled (default), Gateway advertises Fetch up to v12 duringApiVersions negotiation to force topic-name fetches; when enabled, clients can negotiate Fetch v13 or higher. Additional APIs will land behind the same flag in upcoming releases.
Enable by setting GATEWAY_FEATURE_FLAGS_TOPIC_ID_SUPPORT=true. You also have to set GATEWAY_FEATURE_FLAGS_INCREMENTAL_FETCH_SESSIONS_ENABLED=false, because incremental fetch sessions are enabled by default and Gateway doesn’t start when both flags are set.
This is a preview feature. We recommend enabling it only in non-production environments.
Incremental fetch sessions
Apache Kafka clients can use incremental fetch sessions (KIP-227) to make Fetch requests smaller: a consumer opens a session with a first, full Fetch request, then sends only the partitions that changed on each later request while the broker tracks the rest. Gateway supports these sessions through the proxy and enables them by default. To opt out, setGATEWAY_FEATURE_FLAGS_INCREMENTAL_FETCH_SESSIONS_ENABLED=false.
Incompatibilities
- Topic ID support: you can’t combine incremental fetch sessions with topic ID support. Gateway doesn’t start when both
GATEWAY_FEATURE_FLAGS_INCREMENTAL_FETCH_SESSIONS_ENABLEDandGATEWAY_FEATURE_FLAGS_TOPIC_ID_SUPPORTare set totrue. Incremental fetch sessions are on by default, so to use the topic ID preview you have to setGATEWAY_FEATURE_FLAGS_INCREMENTAL_FETCH_SESSIONS_ENABLED=false. - SQL and CEL topics: these deprecated logical topics are incompatible with incremental fetch sessions and return incorrect results unless you opt out. Topic views aren’t affected.
If you use the audit Interceptor or a fetch policy, incremental fetch sessions change how they behave. See those pages for details.
Licensing
Monitoring
Audit
GATEWAY_AUDIT_LOG_EVENT_TYPES valid value is ALL (default) or a comma-separated list of the following:
CONNECTIONfor initial connection event, i.e. receiving Kafka’sAPI_VERSIONSrequestAUTHENTICATIONfor authentication success or failure eventREST_APIfor all REST v1 admin requests and all REST v2 requests (see API reference)ACL_ADMINfor ACL management events done via Kafka Admin API, i.e.CREATE_ACLSandDELETE_ACLSrequestsAUTHORIZATIONfor authorization failure events, i.e. when ACL permissions deny an operation on a resource
Logging
Product analytics
Advanced configuration
These settings are for advanced tuning and debugging. Most deployments don’t need to change them.Threading
Gateway internal timeout
These timeout thresholds are used by Gateway and typically don’t need modification since they match Apache Kafka’s defaults. If your Kafka brokers or clients use non-default timeout values, you may want to adjust these.Graceful shutdown
During shutdown, Gateway closes client connections in controlled manner to simulate rolling Kafka broker restart, with
GATEWAY_SHUTDOWN_DELAY_BETWEEN_BROKERS_MS pause between brokers. For librdkafka clients, set GATEWAY_SHUTDOWN_DELAY_BETWEEN_BROKERS_MS to 1000 or higher to prevent them from crashing during shutdown because of ALL_BROKERS_DOWN error.
Internal topics
As Gateway is stateless, it uses Kafka topics to store its internal state. Use the following environment variables to configure these internal topics. If missing, Gateway will automatically create the topics (if it has the permission to do so). You can also create the topics independently of Gateway, just make sure they are configured as described below.Internal state
Firstly, there are some general configuration settings for Gateway internal state management which apply to all used topics.Topic names
Required topic configuration
The most important setting islog.cleanup.policy which defines the clean up policy for the topic. Most of the topics used by Gateway are compacted, but some use time-based retention. If this isn’t set up properly, Gateway will throw an error on startup. Set the following:
log.cleanup.policy=compactfor compactionlog.cleanup.policy=deletefor time based retention
Data Quality topic configs
API blocking modes
TheGATEWAY_FEATURE_FLAGS_BLOCK_UNSUPPORTED_APIS environment variable controls how Gateway handles Kafka APIs that aren’t explicitly supported.
Two modes:
- Permissive mode (
false, default): Gateway allows unsupported APIs to pass through - maintains backward compatibility - Restrictive mode (
true): Gateway blocks unsupported APIs for enhanced security
- Use permissive (default) when upgrading or if you have legacy applications
- Use restrictive when you want maximum security and only allow known, supported APIs
false):
- ✅ Backward compatibility: Legacy applications continue to work
- ✅ Smooth upgrades: No breaking changes during Gateway updates
- ⚠️ Security risk: Unsupported APIs bypass Gateway’s security controls
true):
- ✅ Enhanced security: Only explicitly supported APIs are allowed
- ✅ Controlled environment: Prevents access to potentially problematic APIs
- ❌ Breaking changes: Applications using unsupported APIs will fail
- Stability-blocked APIs (always blocked):
- Stability (APIs not yet fully implemented or tested in Gateway):
-
DESCRIBE_TOPIC_PARTITIONS- New API for describing topic partitions -
CONSUMER_GROUP_DESCRIBE- New consumer group protocol API (KIP-848) -
CONSUMER_GROUP_HEARTBEAT- New consumer group protocol API (KIP-848) - New Kafka 4 group protocols (Gateway has no rebuilder for these, so it blocks them for every client):
-
Share groups (KIP-932), also known as Kafka queues:
SHARE_GROUP_HEARTBEAT,SHARE_GROUP_DESCRIBE,SHARE_FETCH,SHARE_ACKNOWLEDGE,INITIALIZE_SHARE_GROUP_STATE,READ_SHARE_GROUP_STATE,WRITE_SHARE_GROUP_STATE,DELETE_SHARE_GROUP_STATE,READ_SHARE_GROUP_STATE_SUMMARY,DESCRIBE_SHARE_GROUP_OFFSETS,ALTER_SHARE_GROUP_OFFSETS,DELETE_SHARE_GROUP_OFFSETS -
Streams groups (KIP-1071), the new Kafka Streams rebalance protocol:
STREAMS_GROUP_HEARTBEAT,STREAMS_GROUP_DESCRIBE - Admin operations (these operations should be performed directly on the underlying Kafka cluster rather than through Gateway):
-
ALTER_REPLICA_LOG_DIRS- Admin operation for altering replica log directories -
ELECT_LEADERS- Admin operation for leader election -
ALTER_PARTITION_REASSIGNMENTS- Admin operation for partition reassignments -
LIST_PARTITION_REASSIGNMENTS- Admin operation for listing partition reassignments -
ALTER_PARTITION- Admin operation for altering partitions -
ALLOCATE_PRODUCER_IDS- Admin operation for producer ID allocation -
CREATE_DELEGATION_TOKEN- Admin operation for creating delegation tokens -
RENEW_DELEGATION_TOKEN- Admin operation for renewing delegation tokens -
EXPIRE_DELEGATION_TOKEN- Admin operation for expiring delegation tokens -
DESCRIBE_DELEGATION_TOKEN- Admin operation for describing delegation tokens -
DESCRIBE_USER_SCRAM_CREDENTIALS- Admin operation for describing SCRAM credentials -
ALTER_USER_SCRAM_CREDENTIALS- Admin operation for altering SCRAM credentials -
DESCRIBE_CLIENT_QUOTAS- Admin operation for describing client quotas -
ALTER_CLIENT_QUOTAS- Admin operation for altering client quotas -
UPDATE_FEATURES- Admin operation for updating cluster features
- Unknown/Unsupported APIs (affected by this setting):
- KIP-1000 Configuration APIs (bypass ACL checks):
LIST_CLIENT_METRICS_RESOURCES(KIP-1000) - List client metrics configuration resources without DESCRIBE_CONFIGS permission
- Internal/Admin APIs (not meant for client use):
LEADER_AND_ISR- Internal leader election and ISR managementSTOP_REPLICA- Internal replica managementUPDATE_METADATA- Internal metadata updatesCONTROLLED_SHUTDOWN- Internal broker shutdown coordinationWRITE_TXN_MARKERS- Internal transaction marker managementDESCRIBE_QUORUM- Internal quorum managementENVELOPE- Internal request envelope handlingUNREGISTER_BROKER- Internal broker registration managementADD_RAFT_VOTER- Internal Raft consensus managementREMOVE_RAFT_VOTER- Internal Raft consensus management
- Future Kafka APIs not yet implemented in Gateway
- Any other APIs that may bypass Gateway’s security controls or don’t work correctly with Gateway features
GATEWAY_FEATURE_FLAGS_BLOCK_UNSUPPORTED_APIS=false (permissive mode):
- Unsupported APIs are passed through to Kafka without Gateway processing
- Configuration exposure: KIP-1000 APIs allow access to sensitive configuration without proper permissions
- Internal API access: Internal/Admin APIs meant for broker-to-broker communication may be accessible to clients
- Topic mapping and other Gateway features may not work correctly
- Potential for unauthorized access to restricted resources and information disclosure
GATEWAY_FEATURE_FLAGS_BLOCK_UNSUPPORTED_APIS=true (restrictive mode):
- All unsupported APIs are blocked at the Gateway level
- Prevents security vulnerabilities: Blocks KIP-1000 APIs that bypass ACL checks
- Prevents internal API access: Blocks internal/Admin APIs that should not be accessible to clients
- Enhanced security through explicit allow-list approach
- Applications must use only Gateway-supported APIs
ListClientMetricsResources API bypasses ACL checks:
When GATEWAY_FEATURE_FLAGS_BLOCK_UNSUPPORTED_APIS=false (permissive mode), clients can call the ListClientMetricsResources API without proper authorization. This API is designed to list all client metrics configuration resources in the cluster.
Attack scenario:
- Unauthorized client connects to Gateway without
DESCRIBE_CONFIGSpermission onCLUSTERresource - Client calls
ListClientMetricsResourcesAPI directly - Gateway forwards the request to Kafka without ACL validation (since it’s not in the supported APIs list)
- Kafka responds with complete list of client metrics configuration resources
- Client receives sensitive configuration information including:
- All configured client metrics resources
- Internal cluster configuration details
- Resource names and configurations that should be protected
- Information disclosure: Unauthorized access to sensitive cluster configuration
- ACL bypass: Complete circumvention of Gateway’s permission system
- Compliance violation: Access to restricted data without proper authorization
- Reconnaissance: Attackers can enumerate cluster resources for further attacks
ListClientMetricsResources API is not included in Gateway’s supported APIs list, so when permissive mode is enabled, it bypasses all Gateway security controls and is forwarded directly to Kafka without ACL validation.
This setting only affects unsupported APIs. APIs that are explicitly blocked for security or stability reasons (like consumer group management) will always be blocked, regardless of this setting.