LSTM is dead. Long Live Transformers!
Leo Dirac (@leopd) talks about how LSTM models for Natural Language Processing (NLP) have been practically replaced by transformer-based models. Basic background on NLP, and a brief history of supervised learning techniques on documents, from bag of words, through vanilla RNNs and LSTM. Then there's a technical deep dive into how Transformers work with multi-headed self-attention, and positional encoding. Includes sample code for applying these ideas to real-world projects.
Видео LSTM is dead. Long Live Transformers! канала Seattle Applied Deep Learning
Видео LSTM is dead. Long Live Transformers! канала Seattle Applied Deep Learning
Показать
Комментарии отсутствуют
Информация о видео
3 декабря 2019 г. 23:40:18
00:28:48
Другие видео канала
Illustrated Guide to Transformers Neural Network: A step by step explanationNatural Language Processing in PythonGeometric Intuition for Training Neural NetworksCS480/680 Lecture 19: Attention and Transformer NetworksDETR: End-to-End Object Detection with Transformers (Paper Explained)Deep Learning State of the Art (2020) | MIT Deep Learning SeriesDeep Generative Modeling | MIT 6.S191Lecture 10 | Recurrent Neural NetworksAn Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (Paper Explained)TransformerTransformer Neural Networks - EXPLAINED! (Attention is all you need)Deep Learning 7. Attention and Memory in Deep LearningHopfield Networks is All You Need (Paper Explained)Attention is all you need; Attentional Neural Network Models | Łukasz Kaiser | MasterclassBERT Neural Network - EXPLAINED!18- Long Short Term Memory (LSTM) Networks Explained EasilyBut what is a Neural Network? | Deep learning, chapter 1MIT Introduction to Deep Learning | 6.S191Deep Learning: A Crash CourseGPT-3: Language Models are Few-Shot Learners (Paper Explained)