Skip to main content
Quick navigation

Simulate Broken Brokers

This interceptor injects intermittent errors in client connections to brokers that are consistent with broker side issues.

This demo will run you through some of these use cases step-by-step.

View the full demo in realtime​

You can either follow all the steps manually, or watch the recording

Review the docker compose environment​

As can be seen from docker-compose.yaml the demo environment consists of the following services:

  • gateway1
  • gateway2
  • kafka-client
  • kafka1
  • kafka2
  • kafka3
  • schema-registry
cat docker-compose.yaml

Starting the docker environment​

Start all your docker processes, wait for them to be up and ready, then run in background

  • --wait: Wait for services to be running|healthy. Implies detached mode.
  • --detach: Detached mode: Run containers in the background
docker compose up --detach --wait

Creating topic my-topic on gateway1​

Creating on gateway1:

  • Topic my-topic with partitions:1 and replication-factor:1
kafka-topics \
    --bootstrap-server localhost:6969 \
    --replication-factor 1 \
    --partitions 1 \
    --create --if-not-exists \
    --topic my-topic

Adding interceptor simulate-broken-brokers​

Let's create the interceptor, instructing Conduktor Gateway to inject failures for some Produce requests that are consistent with broker side issues.

step-06-simulate-broken-brokers-interceptor.json:

{
  "kind" : "Interceptor",
  "apiVersion" : "gateway/v2",
  "metadata" : {
    "name" : "simulate-broken-brokers"
  },
  "spec" : {
    "comment" : "Adding interceptor: simulate-broken-brokers",
    "pluginClass" : "io.conduktor.gateway.interceptor.chaos.SimulateBrokenBrokersPlugin",
    "priority" : 100,
    "config" : {
      "rateInPercent" : 100,
      "errorMap" : {
        "FETCH" : "UNKNOWN_SERVER_ERROR",
        "PRODUCE" : "CORRUPT_MESSAGE"
      }
    }
  }
}
curl \
    --silent \
    --request PUT "http://localhost:8888/gateway/v2/interceptor" \
    --header "Content-Type: application/json" \
    --user "admin:conduktor" \
    --data @step-06-simulate-broken-brokers-interceptor.json | jq

Listing interceptors​

Listing interceptors on gateway1

curl \
    --silent \
    --request GET "http://localhost:8888/gateway/v2/interceptor" \
    --user "admin:conduktor" | jq

Let's produce some records to our created topic and observe some errors being injected by Conduktor Gateway.​

This should produce output similar to this:

[2023-12-19 14:08:09,150] WARN [Producer clientId=producer-1] Got error produce response with correlation id 3 on topic-partition my-topic-0, retrying (1 attempts left). Error: CORRUPT_MESSAGE (org.apache.kafka.clients.producer.internals.Sender)
[2023-12-19 14:08:09,252] WARN [Producer clientId=producer-1] Got error produce response with correlation id 4 on topic-partition my-topic-0, retrying (1 attempts left). Error: CORRUPT_MESSAGE (org.apache.kafka.clients.producer.internals.Sender)
kafka-producer-perf-test \
  --producer-props \
      bootstrap.servers=localhost:6969 \
      retries=5 \
  --record-size 10 \
  --throughput 1 \
  --num-records 10 \
  --topic my-topic

Remove interceptor simulate-broken-brokers​

Let's delete the interceptor simulate-broken-brokers so we can stop chaos injection

curl \
    --silent \
    --request DELETE "http://localhost:8888/gateway/v2/interceptor/simulate-broken-brokers" \
    --header "Content-Type: application/json" \
    --user "admin:conduktor" \
    --data-raw '{
  "vCluster" : "passthrough"
}' | jq

Listing interceptors​

Listing interceptors on gateway1

curl \
    --silent \
    --request GET "http://localhost:8888/gateway/v2/interceptor" \
    --user "admin:conduktor" | jq

Let's produce some records to our created topic with no chaos​

kafka-producer-perf-test \
  --producer-props bootstrap.servers=localhost:6969 \
  --record-size 10 \
  --throughput 1 \
  --num-records 10 \
  --topic my-topic

Tearing down the docker environment​

Remove all your docker processes and associated volumes

  • --volumes: Remove named volumes declared in the "volumes" section of the Compose file and anonymous volumes attached to containers.
docker compose down --volumes

Conclusion

Yes, Chaos Simulate Broken Brokers is simple as it!