Linear algebra with Transformers – Paper Explained

Why would one build a transformer to solve linear algebra problems when there is numpy.linalg? Check out the video to find out why this is a cool idea and understand how the transformer works that can solve 9 linear algebra problems (e.g. matrix multiplication, inversion).
► SPONSOR: Weights & Biases 👉 https://wandb.me/ai-coffee-break

❓ Quiz Questions: https://www.youtube.com/c/AICoffeeBreak/community
➡️ AI Coffee Break Merch! 🛍️ https://aicoffeebreak.creator-spring.com/

📺 Symbolic Mathematics with transformers: https://youtu.be/l7ofrfmVsd0
📺 Transformer explained: https://youtube.com/playlist?list=PLpZBeKTZRGPNdymdEsSSSod5YQ3Vu0sKY
📺 GPT-3: https://youtu.be/5fqxPOaaqi0
📺 Foundation models: https://youtu.be/4Cxz4rnnZ7Q
📺 Interpolation vs. Generalization in Deep Learning: https://youtu.be/-GH9vW5FWUs

Thanks to our Patrons who support us in Tier 2, 3, 4: 🙏
donor, Dres. Trost GbR, Yannik Schneider

Paper 📜: Charton, François. "Linear algebra with transformers." arXiv preprint arXiv:2112.01898 (2021). https://arxiv.org/abs/2112.01898
🔗 Openreview discussion between author and reviewers: https://openreview.net/forum?id=L2a_bcarHcF
🔗 A cat can be an author, too! https://en.wikipedia.org/wiki/F._D._C._Willard
🔗 Gary Marcus: extrapolationhttps://twitter.com/GaryMarcus/status/1411401507610796032

▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔥 Optionally, pay us a coffee to help with our Coffee Bean production! ☕
Patreon: https://www.patreon.com/AICoffeeBreak
Ko-fi: https://ko-fi.com/aicoffeebreak
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀

Outline:
00:00 Linear algebra with transformers
00:41 Weights & Biases (Sponsor)
02:21 Why throwing transformers at linear algebra is cool.
08:08 How do transformers solve linear algebra?
09:50 Encoding matrices for transformers
11:28 Training data and results
12:43 Generalization!?
16:05 Few-shot learning!?
17:36 AI Coffee Break Quiz call to action

Music 🎵 : Secret Job – Godmode
Cat meowing sound by ignotus : https://freesound.org/people/ignotus/sounds/26104/

----------------------------------
🔗 Links:
AICoffeeBreakQuiz: https://www.youtube.com/c/AICoffeeBreak/community
Twitter: https://twitter.com/AICoffeeBreak
Reddit: https://www.reddit.com/r/AICoffeeBreak/
YouTube: https://www.youtube.com/AICoffeeBreak

#AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #research​

Видео Linear algebra with Transformers – Paper Explained канала AI Coffee Break with Letitia