Pre-requisite Docker Compose
Overview
Once deployed, you’ll have access to a fully functional Kafka stack integrated with Conduktor’s UI and backend services. This environment is ideal for trying out Conduktor’s features before deploying to a production environment.
1
The data generation service publishes a continuous stream of synthetic events that simulate an e-commerce business.
2
Conduktor Gateway intercepts Kafka traffic to apply real-time operational controls — including encryption, decryption, data masking, and Traffic Control Policies — without requiring changes to producers or consumers.
3
The intercepted data flows through the Conduktor Gateway proxy and lands in a Redpanda Kafka cluster with a schema registry enforcing structure and compatibility across topics.
4
Conduktor Console provides a centralized UI-based control plane to manage both the Redpanda Kafka cluster and Conduktor Gateway, allowing you to:
- monitor topics, schemas, consumer groups and connectors
- define and apply Gateway policies
- trace message flows
- audit activity across the entire stack
Data policies
The data policies configured in this get-started stack are:| Policy | Description |
|---|---|
| ProducePolicyPlugin | Ensures produced messages adhere to configuration requirements. Parameters configure Block acks -1 and audit compression (GZIP, ZSTD, etc). |
| EncryptPlugin | Encrypts full payloads for topics matching pattern: .*_encrypted$. Check the customers_encrypted topic in the local-kafka cluster. |
| DecryptPlugin | Decrypts payloads on consume for topics matching .*_encrypted$. Shows plaintext in cdk-gateway. |
| FieldLevelDataMaskingPlugin | Masks profile.creditCardNumber, contact.email and contact.phone in topics matching pattern: ^[A-Za-z]*_masked$. |
| CreateTopicPolicyPlugin | Validates topic config: partitions have to be 1–3 or creation is Blocked. |