Getting Started with Python

Kafka with Docker Compose

The fastest way to get Kafka running locally for development. This uses KRaft mode (no ZooKeeper needed):

# docker-compose.yml
version: '3.8'
services:
  kafka:
    image: confluentinc/cp-kafka:7.6.0
    ports:
      - "9092:9092"
    environment:
      KAFKA_NODE_ID: 1
      KAFKA_PROCESS_ROLES: broker,controller
      KAFKA_CONTROLLER_QUORUM_VOTERS: 1@kafka:29093
      KAFKA_LISTENERS: PLAINTEXT://0.0.0.0:9092,CONTROLLER://0.0.0.0:29093
      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
      KAFKA_CONTROLLER_LISTENER_NAMES: CONTROLLER
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
      KAFKA_LOG_DIRS: /var/lib/kafka/data
      CLUSTER_ID: 'MkU3OEVBNTcwNTJENDM2Qk'

# Start Kafka
docker-compose up -d

# Verify it's running
docker-compose exec kafka kafka-topics --bootstrap-server localhost:9092 --list

# Create a test topic
docker-compose exec kafka kafka-topics --bootstrap-server localhost:9092 \
  --create --topic test --partitions 3 --replication-factor 1

# Stop and clean up
docker-compose down -v

Alternative: Redpanda

Redpanda is a Kafka-compatible streaming platform written in C++. It's simpler to run locally (single binary, no JVM) and API-compatible with Kafka. Great for local development: docker run -d --name redpanda -p 9092:9092 redpandadata/redpanda

Key Takeaway: Use KRaft mode (no ZooKeeper) for local development. Use KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 for single-broker dev setups. Never use replication-factor 1 in production.

Practice Exercises

Medium Build a Mini Project

Combine concepts from this tutorial to build a small utility or tool.

Medium Debug Challenge

Introduce a bug in one of the code examples and practice finding and fixing it.

Hard Refactoring Exercise

Rewrite one example using a different approach and compare the tradeoffs.