​
Overview

SNI routing reduces the number of Gateway ports exposed to clients. Unlike port mapping, which differentiates brokers based on the port a client connects to, SNI routing distinguishes brokers by the hostname the client wants to connect to. This allows Gateway to expose a single port while directing traffic to the appropriate broker based on the client’s requested hostname. This is particularly useful if you are:
  • Experiencing a high administrative overhead for managing multiple ports.
  • Restricted by your security department in the number of ports you can expose.

​
Context about client connection

To understand the different steps required to set up SNI routing, let’s break it down into the connection steps each client would take:
  1. Connect to Gateway advertised host
  2. Retrieve how to reach the right broker for each topic partition in the metadata returned by Gateway
  3. Directly connect to the desired broker through Gateway
This means that we will have to:
In order to keep Gateway as stateless as possible, we do not store the metadata internally. We simply pass it on to the client. This means that the metadata will be refreshed every time a client asks for it (e.g. when a new client connects or when the connection refreshes).

​
Set up SNI routing

​
1. Define ports

With Gateway using SNI routing, you only expose a single port for all your brokers. However, depending on your infrastructure, you may want to differentiate between the port the Gateway container listens on and the port returned in the metadata for clients to connect. This could be the case if you’re using a load balancer: SNI ports By default, Gateway listens on port 6969. This port can be configured using the GATEWAY_PORT_START environment variable. To configure the port that is returned in the metadata, you can use the GATEWAY_ADVERTISED_SNI_PORT. By default, this port will be the same as the GATEWAY_PORT_START.

​
2. TLS termination

When Gateway is set up to act as the SNI router, it has to terminate (i.e., decrypt) the incoming TLS connections because:
  1. SNI information requires TLS handling: this information is transmitted in the TLS ClientHello handshake. To forward network requests based on TLS SNI, Gateway has to participate in the TLS handshake to extract the server name indication. If TLS is terminated upstream of Gateway, the SNI information is no longer available.
  2. Kafka protocol inspection requires decryption: even though SNI information is available in the un-encrypted portion of the TLS handshake, Gateway has to decrypt the traffic to apply its functionality, which operates on the Kafka protocol layer. Since Gateway needs to inspect, modify or route based on Kafka protocol messages (API keys, topics, etc.), TLS passthrough is not possible.
Therefore, Gateway has to both:
  • handle the TLS handshake with the client and
  • have a valid keystore certificate for the advertised host (and all the brokers in the cluster).

​
3. Prepare Gateway’s keystore certificate

The keystore certificate for Gateway with SNI routing needs to include the Gateway advertised host, as well as a SAN for each advertised broker in the cluster. Alternatively, wildcards * can be used in the SAN, if supported by your issuer and security team. If you need to detail all the advertised brokers in the Gateway keystore, here is the format returned by Gateway for each broker: Advertised brokers We can find:
  • The advertised host prefix, which is customizable with GATEWAY_ADVERTISED_HOST_PREFIX and defaults to broker
  • The Kafka cluster ID in Gateway, which is not customizable and defaults to main.
  • The broker ID in the Kafka cluster, which is unique for each broker and retrieved directly from your Kafka cluster.
  • The SNI host separator, which is customizable with GATEWAY_SNI_HOST_SEPARATOR and defaults to -.
  • The advertised host, which is customizable with GATEWAY_ADVERTISED_HOST.
Example: Let’s say we want to advertise Gateway as gateway.conduktor.sni-demo.local and we have a Kafka cluster with 3 brokers with IDs 1, 2, and 3. The SANs for the certificate would be: 
gateway.conduktor.sni-demo.local
brokermain1-gateway.conduktor.sni-demo.local
brokermain2-gateway.conduktor.sni-demo.local
brokermain3-gateway.conduktor.sni-demo.local
Another option is to use wildcards in the SANs, for example: 
*.conduktor.sni-demo.local

​
4. Configure DNS

Next, create DNS entries to allow clients to be properly routed to the Gateway advertised host and Kafka brokers through Gateway. 
gateway.conduktor.sni-demo.local
brokermain1-gateway.conduktor.sni-demo.local
brokermain2-gateway.conduktor.sni-demo.local
brokermain3-gateway.conduktor.sni-demo.local

​
5. Configure Gateway

Here’s the minimum configurations required, depending on the security protocol you want to use for your clients to connect to Gateway.
This is in addition to the KAFKA_ properties required for Gateway to connect to the Kafka cluster.Please check the list of environment variables for Gateway SSL configuration and Gateway SNI routing configuration.
GATEWAY_ROUTING_MECHANISM: host
GATEWAY_ADVERTISED_HOST: gateway.conduktor.sni-demo.local
GATEWAY_ADVERTISED_HOST_PREFIX: broker
GATEWAY_SECURITY_PROTOCOL: SASL_SSL
GATEWAY_SECURITY_MODE: GATEWAY_MANAGED
GATEWAY_SSL_KEY_STORE_PATH: /security/kafka.gateway.conduktor.sni-demo.local.keystore.jks
GATEWAY_SSL_KEY_STORE_PASSWORD: conduktor
GATEWAY_SSL_KEY_PASSWORD: conduktor

​
Debug and troubleshoot

You can use Console’s Brokers tab to view the advertised listeners of a Gateway once the initial connection and authentication are successful: Console Brokers view Alternatively, kcat’s metadata list mode (-L) can be used to check whether the right advertised listeners have been configured or how many brokers are in a given cluster. Setting LOG4J2_IO_CONDUKTOR_PROXY_NETWORK_LEVEL to DEBUG might be helpful when debugging issues. For more details, see a comprehensive SNI routing troubleshooting guide