Beyond neural scaling laws – Paper Explained

„Beyond neural scaling laws: beating power law scaling via data pruning” paper explained with animations. You do not need to train your neural network on the entire dataset!
Sponsor: NVIDIA ❗ use this link to register for the GTC 👉 https://nvda.ws/3p4E5K8
Google Form to enter DLI credits giveaway: https://forms.gle/rkZS4u5rfyySr6iP9

ERRATUM: See pinned comment for what easy/hard examples are chosen.

📺 PaLM model explained: https://youtu.be/yi-A0kWXEO4

Check out our daily #MachineLearning Quiz Questions: https://www.youtube.com/c/AICoffeeBreak/community
Paper 📜: Sorscher, Ben, Robert Geirhos, Shashank Shekhar, Surya Ganguli, and Ari S. Morcos. "Beyond neural scaling laws: beating power law scaling via data pruning." arXiv preprint arXiv:2206.14486 (2022). https://arxiv.org/abs/2206.14486

Outline:
00:00 Stable Diffusion is a Latent Diffusion Model
01:43 NVIDIA (sponsor): Register for the GTC!
03:00 What are neural scaling laws? Power laws explained.
05:15 Exponential scaling in theory
07:40 What the theory predicts
09:50 Unsupervised data pruning with foundation models

Thanks to our Patrons who support us in Tier 2, 3, 4: 🙏
Don Rosenthal, Dres. Trost GbR, Julián Salazar, Edvard Grødem, Vignesh Valliappan, Mutual Information, Mike Ton
➡️ AI Coffee Break Merch! 🛍️ https://aicoffeebreak.creator-spring.com/

▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔥 Optionally, pay us a coffee to help with our Coffee Bean production! ☕
Patreon: https://www.patreon.com/AICoffeeBreak
Ko-fi: https://ko-fi.com/aicoffeebreak
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀

🔗 Links:
AICoffeeBreakQuiz: https://www.youtube.com/c/AICoffeeBreak/community
Twitter: https://twitter.com/AICoffeeBreak
Reddit: https://www.reddit.com/r/AICoffeeBreak/
YouTube: https://www.youtube.com/AICoffeeBreak

#AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #research​

Video editing: Nils Trost

Видео Beyond neural scaling laws – Paper Explained канала AI Coffee Break with Letitia