Day 87: Async PDF Generator — Redis Queue, Workers & Live Dashboard #datastructures #dsa #pdf

In Day 87 of the System Design series, we build an asynchronous PDF generation system that keeps your API fast under load by offloading CPU-heavy work to background workers.

Most tutorials generate PDFs synchronously inside the web request. That works for a few users, but at scale it becomes a silent bottleneck — blocking threads, spiking memory, and slowing down your entire service. This demo shows the production pattern: decouple, queue, and scale workers independently.

What you'll see in this demo
Flask Web API (/generate-pdf) — accepts a PDF request, pushes a job to Redis, and returns a job_id immediately (HTTP 202)
Redis message queue (pdf_jobs_queue) — buffers jobs FIFO so bursts don't crash the API
Dedicated Worker — pulls jobs with BRPOP, generates PDFs with fpdf2, saves to pdf_storage/
Job state tracking — QUEUED → PROCESSING → COMPLETED in jobs.log
Live Dashboard — real-time metrics: queued, processing, completed, failed, PDF count, queue depth
User operations — custom PDF, invoice, report, batch load, burst, and month-end spike simulation
Idempotency — worker skips re-generation if a PDF for the same job_id already exists
Architecture flow
Client → Web API → Redis Queue → Worker → pdf_storage/
The API stays lean and responsive. Workers scale separately when demand spikes — exactly how high-traffic systems handle invoices, reports, and shipping labels at scale.

Видео Day 87: Async PDF Generator — Redis Queue, Workers & Live Dashboard #datastructures #dsa #pdf канала SystemDesign Demo 1