Skip to main content

​
Overview

SNI routing reduces the number of Gateway ports exposed to clients.
Per-listener configuration with GATEWAY_LISTENER_<NAME>_* variables is available from Gateway 3.20.0. On earlier versions, networking is configured with the global variables described in Legacy network configuration. To move an existing deployment over, see Migrate to listener configuration.
Unlike port mapping, which differentiates brokers based on the port a client connects to, SNI routing distinguishes brokers by the hostname the client wants to connect to. This allows Gateway to expose a single port while directing traffic to the appropriate broker based on the client’s requested hostname. This is particularly useful if you are:
  • Experiencing a high administrative overhead for managing multiple ports.
  • Restricted by your security department in the number of ports you can expose.

​
Context about client connection

To understand the different steps required to set up SNI routing, let’s break it down into the connection steps each client would take:
  1. Connect to Gateway advertised host
  2. Retrieve how to reach the right broker for each topic partition in the metadata returned by Gateway
  3. Directly connect to the desired broker through Gateway
This means that we will have to:
In order to keep Gateway as stateless as possible, we do not store the metadata internally. We simply pass it on to the client. This means that the metadata will be refreshed every time a client asks for it (e.g. when a new client connects or when the connection refreshes).

​
Set up SNI routing

​
1. Define ports

With Gateway using SNI routing, you only expose a single port for all your brokers. However, depending on your infrastructure, you may want to differentiate between the port the Gateway container listens on and the port returned in the metadata for clients to connect. This could be the case if you’re using a load balancer: SNI ports Configure the local port and the advertised port using the GATEWAY_LISTENER_<NAME>_PORTS environment variable. With SNI routing, the same single port serves all brokers — the advertised:local format lets you decouple what Gateway listens on locally from what it returns in metadata. For example, to listen locally on 9192 and advertise port 443 to clients (the load balancer’s port): 
GATEWAY_LISTENER_DEFAULT_PORTS=443:9192
If the advertised and local ports are the same, set a single value: 
GATEWAY_LISTENER_DEFAULT_PORTS=9192

​
2. TLS termination

When Gateway is set up to act as the SNI router, it has to terminate (i.e., decrypt) the incoming TLS connections because:
  1. SNI information requires TLS handling: this information is transmitted in the TLS ClientHello handshake. To forward network requests based on TLS SNI, Gateway has to participate in the TLS handshake to extract the server name indication. If TLS is terminated upstream of Gateway, the SNI information is no longer available.
  2. Kafka protocol inspection requires decryption: even though SNI information is available in the un-encrypted portion of the TLS handshake, Gateway has to decrypt the traffic to apply its functionality, which operates on the Kafka protocol layer. Since Gateway needs to inspect, modify or route based on Kafka protocol messages (API keys, topics, etc.), TLS passthrough is not possible.
Therefore, Gateway has to both:
  • handle the TLS handshake with the client and
  • have a valid keystore certificate for the advertised host (and all the brokers in the cluster).

​
3. Prepare Gateway’s keystore certificate

The keystore certificate for Gateway with SNI routing has to include the Gateway bootstrap host, as well as a SAN for each broker in the cluster. Alternatively, wildcards * can be used in the SAN, if supported by your issuer and security team. Gateway derives broker hostnames from GATEWAY_LISTENER_<NAME>_ADVERTISED_HOST_PATTERN using template variables:
TemplateDescriptionExample value
{{nodeId}}Broker ID1, 2, 3
{{physicalCluster}}Backend cluster namemain
{{virtualCluster}}Virtual cluster namemy-partner
{{advertisedHost}}Listener’s advertised hostgateway.conduktor.sni-demo.local
If you don’t set the pattern, Gateway uses the default broker pattern broker-{{physicalCluster}}-{{nodeId}}.{{advertisedHost}} and the default bootstrap pattern bootstrap-{{physicalCluster}}.{{advertisedHost}}. If you set ADVERTISED_HOST_PATTERN but not BOOTSTRAP_HOST_PATTERN, Gateway derives the bootstrap hostname from your advertised pattern by removing the {{nodeId}} segment. Example: To advertise Gateway as gateway.conduktor.sni-demo.local with a Kafka cluster of three brokers (IDs 1, 2, 3) and the default pattern, the SANs for the certificate are: 
bootstrap-main.gateway.conduktor.sni-demo.local
broker-main-1.gateway.conduktor.sni-demo.local
broker-main-2.gateway.conduktor.sni-demo.local
broker-main-3.gateway.conduktor.sni-demo.local
The first entry is the bootstrap hostname; the next three are the per-broker hostnames. Another option is to use wildcards in the SANs, for example: 
*.gateway.conduktor.sni-demo.local

​
4. Configure DNS

Next, create DNS entries to allow clients to be properly routed to the Gateway advertised host and Kafka brokers through Gateway. All entries have to point to the Gateway IP (or load balancer). On AWS, use Route 53. You can either list each hostname individually, or use a wildcard record if your DNS provider supports it. Per-broker entries: 
bootstrap-main.gateway.conduktor.sni-demo.local
broker-main-1.gateway.conduktor.sni-demo.local
broker-main-2.gateway.conduktor.sni-demo.local
broker-main-3.gateway.conduktor.sni-demo.local
Wildcard entry: 
*.gateway.conduktor.sni-demo.local
A single wildcard record covers the bootstrap hostname and all broker hostnames. Note that a standard wildcard matches one subdomain level only, so this works as long as your hostnames are all direct subdomains of the same domain.

​
5. Configure Gateway

Here’s the minimum configuration required, depending on the security protocol you want clients to use when connecting to Gateway. The examples below define a single listener named DEFAULT.
This is in addition to the KAFKA_ properties required for Gateway to connect to the Kafka cluster.Check the full reference for Gateway SSL configuration and listener environment variables.
GATEWAY_SECURITY_MODE: GATEWAY_MANAGED
GATEWAY_ACL_ENABLED: true
GATEWAY_LISTENER_DEFAULT_SECURITY_PROTOCOL: SASL_SSL
GATEWAY_LISTENER_DEFAULT_ROUTING: sni
GATEWAY_LISTENER_DEFAULT_PORTS: 9092
GATEWAY_LISTENER_DEFAULT_ADVERTISED_HOST: gateway.conduktor.sni-demo.local
GATEWAY_SSL_KEY_STORE_PATH: /security/kafka.gateway.conduktor.sni-demo.local.keystore.jks
GATEWAY_SSL_KEY_STORE_PASSWORD: conduktor
GATEWAY_SSL_KEY_PASSWORD: conduktor

​
Debug and troubleshoot

When you add a new broker, clients receive its SNI hostname (e.g. brokermain4-gateway.conduktor.sni-demo.local) in the metadata returned by Gateway. For clients to connect successfully, that hostname has to resolve to Gateway and be covered by Gateway’s TLS certificate.
What to updateWith wildcardWithout wildcard
DNSNothing. The wildcard already covers the new hostname.Add an A record for the new broker hostname, pointing to the Gateway IP.
Gateway keystore certificateNothing. The wildcard already covers the new hostname.Add the new broker hostname as a SAN, then renew and redeploy the certificate (see below).
Client truststoreNothing.Nothing. Clients trust the Gateway certificate, not individual brokers.
Gateway restartNothing.Not required. Gateway hot-reloads the keystore every five minutes by default. Configure the interval with GATEWAY_SSL_UPDATE_CONTEXT_INTERVAL_MINUTES.
Redeploying the certificate (without wildcard):Gateway reads the keystore from the file path set in GATEWAY_SSL_KEY_STORE_PATH. How you update it depends on your secrets management setup:
  • Kubernetes secret mounted as a volume: update the secret and Kubernetes will sync the file on disk (typically within one to two minutes). Gateway picks it up on its next reload cycle, within the configured interval.
  • Vault Agent injecting a file: Vault Agent writes the renewed certificate to disk when it detects a rotation. Gateway picks it up on its next reload cycle, within the configured interval.
In both cases, no restart is needed. New connections after the next reload will use the updated certificate. Existing connections keep their originally negotiated TLS context.
Adding a new Kafka broker propagates through the entire chain automatically. Each Gateway updates its in-memory broker topology on every metadata response, discovers the new broker, and generates its SNI hostname without any configuration change.However, DNS and certificate coverage have to be in place at every layer for connections to succeed.
What to updateWith wildcardWithout wildcard
Gateway 1 DNS + keystoreNothing.Add the new broker hostname, as described above.
Gateway 2 DNS + keystoreNothing.Add the new broker hostname for Gateway 2’s own advertised hostnames.
Gateway 2 upstream truststoreNothing.On the Gateway 2 → Gateway 1 connection, Gateway 2 validates Gateway 1’s keystore certificate against Gateway 2’s truststore. If Gateway 1’s keystore doesn’t cover the new broker’s SNI hostname, that connection will fail. Make sure Gateway 1’s updated keystore certificate is trusted by Gateway 2 (KAFKA_SSL_TRUSTSTORE_LOCATION on Gateway 2).
Adding a new Gateway 1 pod has no impact. Gateway instances are stateless with respect to broker topology: broker IDs come from the upstream Kafka cluster and are identical across all pods. Adding a pod does not introduce new broker IDs or hostnames.
You can use Console’s Brokers tab to view the advertised listeners of a Gateway once the initial connection and authentication are successful:Console Brokers viewAlternatively, kcat ‘s metadata list mode (-L) can be used to check whether the right advertised listeners have been configured or how many brokers are in a given cluster.Setting LOG4J2_IO_CONDUKTOR_PROXY_NETWORK_LEVEL to DEBUG might be helpful when debugging issues.