Skip to main content

​
Overview

SNI routing reduces the number of Gateway ports exposed to clients. Unlike port mapping, which differentiates brokers based on the port a client connects to, SNI routing distinguishes brokers by the hostname the client wants to connect to. This allows Gateway to expose a single port while directing traffic to the appropriate broker based on the client’s requested hostname. This is particularly useful if you are:
  • Experiencing a high administrative overhead for managing multiple ports.
  • Restricted by your security department in the number of ports you can expose.

​
Context about client connection

To understand the different steps required to set up SNI routing, let’s break it down into the connection steps each client would take:
  1. Connect to Gateway advertised host
  2. Retrieve how to reach the right broker for each topic partition in the metadata returned by Gateway
  3. Directly connect to the desired broker through Gateway
This means that we will have to:
In order to keep Gateway as stateless as possible, we do not store the metadata internally. We simply pass it on to the client. This means that the metadata will be refreshed every time a client asks for it (e.g. when a new client connects or when the connection refreshes).

​
Set up SNI routing

​
1. Define ports

With Gateway using SNI routing, you only expose a single port for all your brokers. However, depending on your infrastructure, you may want to differentiate between the port the Gateway container listens on and the port returned in the metadata for clients to connect. This could be the case if you’re using a load balancer: SNI ports By default, Gateway listens on port 6969. This port can be configured using the GATEWAY_PORT_START environment variable. To configure the port that is returned in the metadata, you can use the GATEWAY_ADVERTISED_SNI_PORT. By default, this port will be the same as the GATEWAY_PORT_START.

​
2. TLS termination

When Gateway is set up to act as the SNI router, it has to terminate (i.e., decrypt) the incoming TLS connections because:
  1. SNI information requires TLS handling: this information is transmitted in the TLS ClientHello handshake. To forward network requests based on TLS SNI, Gateway has to participate in the TLS handshake to extract the server name indication. If TLS is terminated upstream of Gateway, the SNI information is no longer available.
  2. Kafka protocol inspection requires decryption: even though SNI information is available in the un-encrypted portion of the TLS handshake, Gateway has to decrypt the traffic to apply its functionality, which operates on the Kafka protocol layer. Since Gateway needs to inspect, modify or route based on Kafka protocol messages (API keys, topics, etc.), TLS passthrough is not possible.
Therefore, Gateway has to both:
  • handle the TLS handshake with the client and
  • have a valid keystore certificate for the advertised host (and all the brokers in the cluster).

​
3. Prepare Gateway’s keystore certificate

The keystore certificate for Gateway with SNI routing needs to include the Gateway advertised host, as well as a SAN for each advertised broker in the cluster. Alternatively, wildcards * can be used in the SAN, if supported by your issuer and security team. If you need to detail all the advertised brokers in the Gateway keystore, here is the format returned by Gateway for each broker: Advertised brokers We can find:
  • The advertised host prefix, which is customizable with GATEWAY_ADVERTISED_HOST_PREFIX and defaults to broker
  • The Kafka cluster ID in Gateway, which is not customizable and defaults to main.
  • The broker ID in the Kafka cluster, which is unique for each broker and retrieved directly from your Kafka cluster.
  • The SNI host separator, which is customizable with GATEWAY_SNI_HOST_SEPARATOR and defaults to -.
  • The advertised host, which is customizable with GATEWAY_ADVERTISED_HOST.
Example: Let’s say we want to advertise Gateway as gateway.conduktor.sni-demo.local and we have a Kafka cluster with 3 brokers with IDs 1, 2, and 3. The SANs for the certificate would be: 
gateway.conduktor.sni-demo.local
brokermain1-gateway.conduktor.sni-demo.local
brokermain2-gateway.conduktor.sni-demo.local
brokermain3-gateway.conduktor.sni-demo.local
Another option is to use wildcards in the SANs, for example: 
*.conduktor.sni-demo.local

​
4. Configure DNS

Next, create DNS entries to allow clients to be properly routed to the Gateway advertised host and Kafka brokers through Gateway. All entries have to point to the Gateway IP (or load balancer). On AWS, use Route 53. You can either list each hostname individually, or use a wildcard record if your DNS provider supports it. Per-broker entries: 
gateway.conduktor.sni-demo.local
brokermain1-gateway.conduktor.sni-demo.local
brokermain2-gateway.conduktor.sni-demo.local
brokermain3-gateway.conduktor.sni-demo.local
Wildcard entry: 
*.conduktor.sni-demo.local
A single wildcard record covers the bootstrap hostname and all broker hostnames. Note that a standard wildcard matches one subdomain level only, so this works as long as your hostnames are all direct subdomains of the same domain.

​
5. Configure Gateway

Here’s the minimum configurations required, depending on the security protocol you want to use for your clients to connect to Gateway.
This is in addition to the KAFKA_ properties required for Gateway to connect to the Kafka cluster.Please check the list of environment variables for Gateway SSL configuration and Gateway SNI routing configuration.
GATEWAY_ROUTING_MECHANISM: host
GATEWAY_ADVERTISED_HOST: gateway.conduktor.sni-demo.local
GATEWAY_ADVERTISED_HOST_PREFIX: broker
GATEWAY_SECURITY_PROTOCOL: SASL_SSL
GATEWAY_SECURITY_MODE: GATEWAY_MANAGED
GATEWAY_SSL_KEY_STORE_PATH: /security/kafka.gateway.conduktor.sni-demo.local.keystore.jks
GATEWAY_SSL_KEY_STORE_PASSWORD: conduktor
GATEWAY_SSL_KEY_PASSWORD: conduktor

​
Debug and troubleshoot

When you add a new broker, clients receive its SNI hostname (e.g. brokermain4-gateway.conduktor.sni-demo.local) in the metadata returned by Gateway. For clients to connect successfully, that hostname has to resolve to Gateway and be covered by Gateway’s TLS certificate.
What to updateWith wildcardWithout wildcard
DNSNothing. The wildcard already covers the new hostname.Add an A record for the new broker hostname, pointing to the Gateway IP.
Gateway keystore certificateNothing. The wildcard already covers the new hostname.Add the new broker hostname as a SAN, then renew and redeploy the certificate (see below).
Client truststoreNothing.Nothing. Clients trust the Gateway certificate, not individual brokers.
Gateway restartNothing.Not required. Gateway hot-reloads the keystore every five minutes by default. Configure the interval with GATEWAY_SSL_UPDATE_CONTEXT_INTERVAL_MINUTES.
Redeploying the certificate (without wildcard):Gateway reads the keystore from the file path set in GATEWAY_SSL_KEY_STORE_PATH. How you update it depends on your secrets management setup:
  • Kubernetes secret mounted as a volume: update the secret and Kubernetes will sync the file on disk (typically within one to two minutes). Gateway picks it up on its next reload cycle, within the configured interval.
  • Vault Agent injecting a file: Vault Agent writes the renewed certificate to disk when it detects a rotation. Gateway picks it up on its next reload cycle, within the configured interval.
In both cases, no restart is needed. New connections after the next reload will use the updated certificate. Existing connections keep their originally negotiated TLS context.
Adding a new Kafka broker propagates through the entire chain automatically. Each Gateway updates its in-memory broker topology on every metadata response, discovers the new broker, and generates its SNI hostname without any configuration change.However, DNS and certificate coverage have to be in place at every layer for connections to succeed.
What to updateWith wildcardWithout wildcard
Gateway 1 DNS + keystoreNothing.Add the new broker hostname, as described above.
Gateway 2 DNS + keystoreNothing.Add the new broker hostname for Gateway 2’s own advertised hostnames.
Gateway 2 upstream truststoreNothing.On the Gateway 2 → Gateway 1 connection, Gateway 2 validates Gateway 1’s keystore certificate against Gateway 2’s truststore. If Gateway 1’s keystore doesn’t cover the new broker’s SNI hostname, that connection will fail. Make sure Gateway 1’s updated keystore certificate is trusted by Gateway 2 (KAFKA_SSL_TRUSTSTORE_LOCATION on Gateway 2).
Adding a new Gateway 1 pod has no impact. Gateway instances are stateless with respect to broker topology: broker IDs come from the upstream Kafka cluster and are identical across all pods. Adding a pod does not introduce new broker IDs or hostnames.
You can use Console’s Brokers tab to view the advertised listeners of a Gateway once the initial connection and authentication are successful:Console Brokers viewAlternatively, kcat ‘s metadata list mode (-L) can be used to check whether the right advertised listeners have been configured or how many brokers are in a given cluster.Setting LOG4J2_IO_CONDUKTOR_PROXY_NETWORK_LEVEL to DEBUG might be helpful when debugging issues.