CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs (May 2026)

Title: CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs (May 2026)
Link: https://arxiv.org/abs/2605.19269
Date: May 20, 2026

Summary:
CODA is a GPU kernel abstraction that optimizes Transformer training by reparameterizing memory-bound operators, such as normalization and activations, as epilogues to high-performance GEMM kernels. By fusing these computations into the matmul's on-chip lifecycle, CODA significantly reduces global memory traffic and improves end-to-end training efficiency.

Key Topics:
- Transformer Optimization
- Kernel Fusion
- GEMM Epilogues
- Memory-bound Operators
- GPU Programming

Chapters:
00:00 - Introducing CODA
01:29 - Solving Memory Bottlenecks
02:34 - Framework Materialization Trap
03:44 - GPU Kernel Anatomy
05:01 - Epilogue Programming Sandbox
05:59 - Five Composable Primitives
07:13 - Mathematical Reparameterization Strategies
08:13 - Optimizing RMS Norm
09:43 - Fusing Pairwise Activations
11:03 - Backward Pass Efficiency
12:32 - RMS Norm Identity Hack
14:03 - LLM Assisted Kernel Writing
15:32 - Performance Benchmark Results
16:47 - Rethinking Graph Compilers
17:54 - Future Hardware-Software Co-Design

Stock video credits:
- BRoll.io - https://www.pexels.com/@brollio
- José Alfredo Munguía Lira - https://www.pexels.com/@rectorretro
- Life Of Pix - https://www.pexels.com/@life-of-pix
- Silviu Din - https://www.pexels.com/@silviu-din-1620549
- Soumya - https://www.pexels.com/@soumya-1446957
- Chandresh Uike - https://www.pexels.com/@chandresh-uike-754623426
- Hoang Nguyen - https://www.pexels.com/@hoang-nguyen-1781933
- Usman AbdulrasheedGambo - https://www.pexels.com/@theonlyabdulla
- Tom Fisk - https://www.pexels.com/@tomfisk
- Dan Cristian Pădureț - https://www.pexels.com/@paduret
- Oleg Gamulinskii - https://www.pexels.com/@oleg-gamulinskii-755060
- Magda Ehlers - https://www.pexels.com/@magda-ehlers-pexels
- Malte Luk - https://www.pexels.com/@maltelu
- olia danilevich - https://www.pexels.com/@olia-danilevich
- fauxels - https://www.pexels.com/@fauxels
- Pavel Danilyuk - https://www.pexels.com/@pavel-danilyuk
- Pressmaster - https://www.pexels.com/@pressmaster
- Max Fischer - https://www.pexels.com/@max-fischer
- Pachon in Motion - https://www.pexels.com/@pachon-in-motion-426015731
- Nicola Narracci - https://www.pexels.com/@nicola-narracci-157460431
- Pixabay - https://www.pexels.com/@pixabay
- tunnel motions - https://www.pexels.com/@tunnelmotions
- Colors Motion Graphics - https://www.pexels.com/@colors-motion-graphics-183847699
- The Instagrapher - https://www.pexels.com/@theinstagrapher
- Yaroslav Shuraev - https://www.pexels.com/@yaroslav-shuraev
- Kindel Media - https://www.pexels.com/@kindelmedia
- Google DeepMind - https://www.pexels.com/@googledeepmind
- Adis Resic - https://www.pexels.com/@adis-resic-297996969
- Colin Jones - https://www.pexels.com/@larchmedia

Видео CODA: Rewriting Transformer Blocks as GEMM-Epilogue Programs (May 2026) канала AI Paper Slop