Kafka CLI

Essential Kafka CLI Commands

The Kafka CLI tools are your primary interface for managing topics, debugging issues, and testing. They ship with every Kafka installation in the bin/ directory. Master these commands and you can diagnose most Kafka issues in minutes.

Topic Management

# Create a topic
kafka-topics.sh --bootstrap-server localhost:9092 \
  --create --topic test-topic --partitions 3 --replication-factor 1

# List all topics
kafka-topics.sh --bootstrap-server localhost:9092 --list

# Describe a topic (partitions, replicas, ISR)
kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic test-topic

# Increase partitions (cannot decrease!)
kafka-topics.sh --bootstrap-server localhost:9092 \
  --alter --topic test-topic --partitions 6

# Delete a topic
kafka-topics.sh --bootstrap-server localhost:9092 --delete --topic test-topic

Producing and Consuming (for testing)

# Console producer — type messages, press Enter to send
kafka-console-producer.sh --bootstrap-server localhost:9092 --topic test-topic
> Hello World
> Another message
> ^C

# Console producer with keys (key:value format)
kafka-console-producer.sh --bootstrap-server localhost:9092 --topic test-topic \
  --property parse.key=true --property key.separator=:
> user1:{"action":"login"}
> user2:{"action":"signup"}

# Console consumer — read messages
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test-topic \
  --from-beginning

# Consumer with key and timestamp display
kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test-topic \
  --from-beginning \
  --property print.key=true \
  --property print.timestamp=true

CreateTime:1705312200000	user1	{"action":"login"}
CreateTime:1705312201000	user2	{"action":"signup"}

Consumer Group Management

# List consumer groups
kafka-consumer-groups.sh --bootstrap-server localhost:9092 --list

# Describe a group (shows lag!)
kafka-consumer-groups.sh --bootstrap-server localhost:9092 \
  --describe --group my-app

# Reset offsets to reprocess data
kafka-consumer-groups.sh --bootstrap-server localhost:9092 \
  --group my-app --topic test-topic \
  --reset-offsets --to-earliest --execute

# Reset to specific timestamp
kafka-consumer-groups.sh --bootstrap-server localhost:9092 \
  --group my-app --topic test-topic \
  --reset-offsets --to-datetime 2024-01-15T00:00:00.000 --execute

kcat (kafkacat) — The Swiss Army Knife

kcat (formerly kafkacat) is a faster, more flexible alternative to the built-in tools:

# Install kcat
brew install kcat  # macOS
apt-get install kafkacat  # Ubuntu

# Produce
echo '{"user":"alice","action":"login"}' | kcat -P -b localhost:9092 -t test-topic

# Consume (last 5 messages)
kcat -C -b localhost:9092 -t test-topic -o -5 -e

# Metadata — show all brokers, topics, partitions
kcat -L -b localhost:9092

# Produce with key
echo 'key1:value1' | kcat -P -b localhost:9092 -t test-topic -K:

Practice Exercises