Day 8: Infrastructure Setup - Docker Multi-Broker Kafka Cluster

The Naive Approach (And Why It Destroys Your Weekend)
You’ve written beautiful H3-based spatial indexing code. You fire up docker run confluentinc/cp-kafka:latest, point your producer at localhost:9092, and start pumping 3,000 driver location updates per second.

It works. For 47 seconds

Then the broker OOMs. Your producer hangs indefinitely. You restart the container—all data is gone. No replication. No persistence. Your carefully crafted DriverLocationUpdate records with their H3 cell indices vanish into the void.

Why this fails:

No replication: Single broker = single point of failure. Restart loses all uncommitted data
No resource limits: Kafka consumes all available memory until Docker kills it.
Networking chaos: localhost:9092 doesn’t work from other containers. advertised.listeners misconfigured.
No Schema Registry: Avro schemas hardcoded in producers/consumers. First schema change breaks everything.
Topic misconfiguration: Default 1 partition can’t handle 3k msg/sec with spatial queries.

Видео Day 8: Infrastructure Setup - Docker Multi-Broker Kafka Cluster канала Hands On Course Demo