Cache Kafka records — Conduktor Gateway

​

Overview

​

Benefits

• Improved performance. By serving fetched records from the cache, subsequent fetch requests can be served faster, reducing the overall latency and improving the response time for clients.
• Reduced load on Kafka cluster. With the cache Interceptor in place, the Kafka cluster experiences reduced load during fetch requests since a portion of the requests can be satisfied from the cache directly, reducing the number of requests hitting the cluster.
• Enhanced scalability. The cache Interceptor provides an additional layer of scalability by distributing the workload between the cache and the Kafka cluster. It can handle a higher volume of fetch requests without overwhelming the Kafka cluster.

​

Configuration options

​

CacheConfig

​

RocksdbConfig

​

InMemConfig

​

Example

curl \
  --request PUT \
  --url 'http://localhost:8888/gateway/v2/interceptor' \
  --header 'Authorization: Basic YWRtaW46Y29uZHVrdG9y' \
  --header 'Content-Type: application/json' \
  --data-raw '{
  "name": "myCacheInterceptor",
  "pluginClass": "io.conduktor.gateway.interceptor.CacheInterceptorPlugin",
  "priority": 100,
  "config": {
    "topic": ".*",
    "cacheConfig": {
      "type": "ROCKSDB",
      "rocksdbConfig": {
        "dbPath": "/caching_storage",
        "cacheSize": 104857600
      }
    }
  }
}'

apiVersion: gateway/v2
kind: Interceptor
metadata:
  name: myCacheInterceptor
  scope:
    vCluster: passthrough
spec:
  pluginClass: io.conduktor.gateway.interceptor.CacheInterceptorPlugin
  priority: 100
  config:
    topic: ".*"
    cacheConfig:
      type: ROCKSDB
      rocksdbConfig:
        dbPath: /caching_storage
        cacheSize: 104857600

conduktor apply -f cache-interceptor.yaml