Pre-requisite Docker Compose
Overview
Once deployed, you’ll have access to a fully functional Kafka stack integrated with Conduktor’s UI and backend services. This environment is ideal for trying out Conduktor’s features before deploying to a production environment.
1
The data generation service publishes a continuous stream of synthetic events that simulate an e-commerce business.
2
Conduktor Gateway intercepts Kafka traffic to apply real-time operational controls — including encryption, decryption, data masking, and Traffic Control Policies — without requiring changes to producers or consumers.
3
The intercepted data flows through the Conduktor Gateway proxy and lands in a Redpanda Kafka cluster with a schema registry enforcing structure and compatibility across topics.
4
Conduktor Console provides a centralized UI-based control plane to manage both the Redpanda Kafka cluster and Conduktor Gateway, allowing you to:
- monitor topics, schemas, consumer groups and connectors
- define and apply Gateway policies
- trace message flows
- audit activity across the entire stack
Data policies
The data policies configured in this get-started stack are:Policy | Description |
---|---|
ProducePolicyPlugin | Ensures produced messages adhere to configuration requirements. Parameters configure Block acks -1 and audit compression (GZIP, ZSTD, etc). |
EncryptPlugin | Encrypts full payloads for topics matching pattern: .*_encrypted$ . Check the customers_encrypted topic in the local-kafka cluster. |
DecryptPlugin | Decrypts payloads on consume for topics matching .*_encrypted$ . Shows plaintext in cdk-gateway . |
FieldLevelDataMaskingPlugin | Masks profile.creditCardNumber , contact.email and contact.phone in topics matching pattern: ^[A-Za-z]*_masked$ . |
CreateTopicPolicyPlugin | Validates topic config: partitions have to be 1–3 or creation is Blocked. |