System Design: Migrating 50TB at 100k RPS (The Hard Way)

Hey everyone. A few months ago, I posted a system design challenge involving a massive 50TB Oracle monolith taking 100,000 writes per second. This video is a deep dive into how you’d actually approach this migration without stopping the world or losing a single bit of data.
We move from first principles—viewing the database not as a static table, but as a transaction log—to building out a full Change Data Capture (CDC) pipeline using Kafka and Debezium. This is "Advanced" system design, so we’ll be going into the weeds on data consistency, idempotency, and the Strangler Fig pattern.

The Log: What every software engineer should know about real-time data's unifying abstraction (Jay Kreps, 2013)
This is the definitive paper on why logs are the heartbeat of distributed systems.

Link: https://engineering.linkedin.com/distributed-systems/log-what-every-software-engineer-should-know-about-real-time-datas-unifying

The Strangler Fig Application (Martin Fowler)
The core pattern for breaking down monoliths incrementally.
Link: https://martinfowler.com/bliki/StranglerFigApplication.html

Debezium Documentation
If you want to see how CDC is implemented in the real world.
Link: https://debezium.io/documentation/

#systemdesign #systemdesigninterview #ai #codingchallenge

Видео System Design: Migrating 50TB at 100k RPS (The Hard Way) канала FAANG Senior Engineer