Kafka consumers can be configured to read from the closest replica rather than always reading from the leader, which can improve performance and reduce cross-datacenter network traffic in geographically distributed deployments.

​
How replica reading works

By default, Kafka consumers always read from the partition leader. However, starting with Kafka 2.4, consumers can be configured to read from follower replicas that are “close” to the consumer.

​
Default behavior (leader-only reads)

  • All consumers read from partition leaders
  • Followers only replicate data, never serve reads
  • Simple and consistent behavior
  • May result in cross-datacenter traffic

​
Closest replica reading

  • Consumers can read from geographically closest replicas
  • Reduces network latency and cross-datacenter bandwidth
  • Requires proper rack awareness configuration
  • Maintains consistency guarantees

​
Configuration

​
Enable closest replica reading

# Consumer configuration
client.rack=us-west-2a

# This tells Kafka which rack/availability zone the consumer is in
# Kafka will prefer replicas in the same rack when available

​
Broker configuration for rack awareness

# Broker configuration (server.properties)
broker.rack=us-west-2a

# Each broker should be configured with its rack/AZ
# This enables Kafka to make intelligent replica placement decisions

​
Topic configuration

When creating topics, consider replica placement: 
# Create topic with rack-aware replica assignment
kafka-topics --bootstrap-server localhost:9092 \
  --create --topic my-topic \
  --partitions 6 \
  --replication-factor 3 \
  --config min.insync.replicas=2

​
Benefits

​
Reduced latency

  • Consumers read from local replicas instead of remote leaders
  • Eliminates cross-datacenter read traffic
  • Lower response times for geographically distributed applications

​
Cost savings

  • Reduces expensive cross-region data transfer
  • Particularly beneficial in cloud environments with data transfer charges
  • Optimizes bandwidth usage in WAN scenarios

​
Improved availability

  • Continues reading even if cross-datacenter links are degraded
  • Better resilience in network partition scenarios
  • Maintains read availability during datacenter issues

​
Consistency considerations

​
Read-after-write consistency

With closest replica reading, you may encounter scenarios where:
  • Producer writes to leader in datacenter A
  • Consumer reads from follower in datacenter B
  • Replication lag may cause temporary inconsistency

​
Mitigation strategies

# Ensure minimum in-sync replicas for writes
min.insync.replicas=2

# Use appropriate acks setting
acks=all

# Configure consumer to handle potential inconsistencies

​
Use cases

​
Multi-region deployments

Region US-West:
- Brokers with rack=us-west
- Consumers with client.rack=us-west
- Reads stay local to us-west replicas

Region EU-Central:
- Brokers with rack=eu-central  
- Consumers with client.rack=eu-central
- Reads stay local to eu-central replicas

​
Availability zone optimization

AZ-1: broker.rack=az-1, client.rack=az-1
AZ-2: broker.rack=az-2, client.rack=az-2
AZ-3: broker.rack=az-3, client.rack=az-3

Each AZ reads from local replicas when possible

​
Monitoring and observability

​
Key metrics to monitor

  • Replica read rates: Track reads from leaders vs followers
  • Cross-datacenter traffic: Monitor reduction in inter-region bandwidth
  • Consumer lag by replica: Ensure follower replicas are keeping up
  • Replication lag: Monitor lag between leader and follower replicas

​
JMX metrics

kafka.server:type=BrokerTopicMetrics,name=FetchMessageConversionsPerSec
kafka.consumer:type=consumer-fetch-manager-metrics,client-id=*,topic=*,partition=*

​
Configuration examples

​
Cloud deployment (AWS)

# Broker configuration
broker.rack=${aws.availability.zone}

# Consumer configuration  
client.rack=us-west-2a

# Additional consumer settings for optimal performance
fetch.min.bytes=1048576
fetch.max.wait.ms=500

​
On-premises multi-datacenter

# Broker configuration
broker.rack=datacenter-1

# Consumer configuration
client.rack=datacenter-1

# Network optimization
socket.receive.buffer.bytes=65536
fetch.max.bytes=52428800

​
Best practices

​
Deployment guidelines

  1. Configure rack awareness on all brokers and consumers
  2. Ensure sufficient replicas in each rack/region
  3. Monitor replication lag to avoid stale reads
  4. Test failover scenarios to ensure proper behavior

​
Performance optimization

# Optimize for closest replica reading
client.rack=your-rack-id
fetch.min.bytes=1024
max.poll.records=500

# Monitor and tune based on your specific latency requirements

​
Consistency requirements

  • Strong consistency needs: Consider leader-only reads
  • Eventually consistent OK: Closest replica reading works well
  • Mixed requirements: Use different consumer groups with different configurations
Replication lag impactWhen reading from follower replicas, consumers may see slightly stale data due to replication lag. Ensure your application can tolerate this eventual consistency model.
Gradual rolloutConsider implementing closest replica reading gradually:
  1. Start with non-critical consumer groups
  2. Monitor metrics and consistency behavior
  3. Expand to more critical workloads as confidence builds

​
Troubleshooting

​
Common issues

  1. Missing rack configuration: Ensure both brokers and consumers have rack settings
  2. Insufficient replicas: Need replicas in consumer’s rack for local reads
  3. High replication lag: Follower replicas falling behind leader
  4. Network configuration: Ensure proper connectivity between racks/regions

​
Verification steps

# Check broker rack configuration
kafka-configs --bootstrap-server localhost:9092 --describe --entity-type brokers

# Monitor consumer metrics
kafka-consumer-groups --bootstrap-server localhost:9092 --describe --group your-group

# Check topic replica distribution
kafka-topics --bootstrap-server localhost:9092 --describe --topic your-topic