kafka-console-producer CLI reads data from standard input and publishes it to Kafka topics. It’s essential for testing, debugging, and quick data injection.
What you’ll learn:
- How to send messages to a Kafka topic
- How to produce messages with keys
- How to produce messages from a file
- Important producer options and settings
How to produce a message into a Kafka topic using the CLI?
To produce to a Kafka topic, we need to provide the mandatory parameters:- Find your Kafka hostname and port e.g.,
localhost:9092 - If Kafka v2.5+, use the
--bootstrap-serveroption - If older version of Kafka, use the
--broker-listoption - Provide the mandatory parameters: topic name
- Use the
kafka-console-producer.shCLI as outlined below
Example
Make sure the topic first_topic is already created with 3 partitions and a replication factor of 1:Ctrl+C.
Command output
> sign. Then any line of text you write afterwards will be sent to the Kafka topic (when pressing Enter)
Gotchas
Here are the common mistakes and caveats with thekafka-console-producer.sh command:
-
Messages are sent with the
nullkey by default (see below for more options) - If the topic does not exist, it can be auto-created by Kafka:
config/server.properties file), with the following defaults:
Extra important options you can set (advanced)
--compression-codec
To enable message compression, default gzip, possible values 'none', 'gzip', 'snappy', 'lz4', or 'zstd'
--producer-property
To pass in any producer property, such as the acks=all setting
--request-required-acks
An alternative to set the acks setting directly
How to produce messages from a file with the Kafka console producer CLI?
Example filetopic-input.txt (make sure each message is on a new line)
How to produce messages with key in the Kafka console producer CLI?
By default messages sent to a Kafka topic will result in messages withnull keys.
We have to use the properties parse.key and key.separator to send the key alongside messages.
In this example, the separator between the key and the value is: :
Quick reference
| Operation | Command |
|---|---|
| Basic produce | kafka-console-producer --bootstrap-server localhost:9092 --topic NAME |
| Produce with keys | ... --property parse.key=true --property key.separator=: |
| With compression | ... --compression-codec snappy |
| With acks=all | ... --producer-property acks=all |
| From file | ... < filename.txt |
See it in practice with ConduktorConduktor Console provides a visual interface for producing messages with full control over headers, keys, and serialization formats.
Next steps
- Consume messages you’ve produced
- Understand message keys and partitioning
- Configure compression for efficiency