The 90ms WebSocket Head-of-Line Bug

Your streamed LLM tokens are stalling 90ms — not the model, not the network. It's head-of-line blocking on a single write buffer. I found it in the trace. Here's the fix and the p99 after. Source: WHATWG Streams + WebSocket framing spec. https://websockets.spec.whatwg.org/

— Dr. Latency

#Shorts

#websocket #llm #streaming

Видео The 90ms WebSocket Head-of-Line Bug канала Driftcache