Week 12 – Practicum: Attention and the Transformer
Course website: http://bit.ly/pDL-home
Playlist: http://bit.ly/pDL-YouTube
Speaker: Alfredo Canziani
Week 12: http://bit.ly/pDL-en-12
0:00:00 – Week 12 – Practicum
PRACTICUM: http://bit.ly/pDL-en-12-3
We introduce attention, focusing on self-attention and its hidden layer representations of the inputs. Then, we introduce the key-value store paradigm and discuss how to represent queries, keys, and values as rotations of an input. Finally, we use attention to interpret the transformer architecture, taking a forward pass through a basic transformer, and comparing the encoder-decoder paradigm to sequential architectures.
0:01:09 – Attention
0:17:36 – Key-value store
0:35:14 – Transformer and PyTorch implementation
0:54:00 – Q&A
Видео Week 12 – Practicum: Attention and the Transformer канала Alfredo Canziani
Playlist: http://bit.ly/pDL-YouTube
Speaker: Alfredo Canziani
Week 12: http://bit.ly/pDL-en-12
0:00:00 – Week 12 – Practicum
PRACTICUM: http://bit.ly/pDL-en-12-3
We introduce attention, focusing on self-attention and its hidden layer representations of the inputs. Then, we introduce the key-value store paradigm and discuss how to represent queries, keys, and values as rotations of an input. Finally, we use attention to interpret the transformer architecture, taking a forward pass through a basic transformer, and comparing the encoder-decoder paradigm to sequential architectures.
0:01:09 – Attention
0:17:36 – Key-value store
0:35:14 – Transformer and PyTorch implementation
0:54:00 – Q&A
Видео Week 12 – Practicum: Attention and the Transformer канала Alfredo Canziani
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
10 – Self / cross, hard / soft attention and the Transformer
An image is worth 16x16 words: ViT | Is this the extinction of CNNs? Long live the Transformer?
NLP for Developers: Transformers | Rasa
Transformer Networks - Attention is all you need easy introduction!
Transformer Neural Networks - EXPLAINED! (Attention is all you need)
Code Review: Transformer - Attention Is All You Need | AISC
07 – Unsupervised learning: autoencoding the targets
Pytorch for Beginners #24 | Transformer Model: Self Attention - Simplest Explanation
Open3D: A Modern Open-Source Library for 3D Data Processing
Rasa Algorithm Whiteboard - Transformers & Attention 3: Multi Head Attention
Gradient descent, how neural networks learn | Chapter 2, Deep learning
Week 9 – Practicum: (Energy-based) Generative adversarial networks
Week 14 – Lecture: Structured prediction with energy based models
Week 10 – Practicum: The Truck Backer-Upper
Developing a deep learning project (case study on transformer)
CS480/680 Lecture 19: Attention and Transformer Networks![[Transformer] Attention Is All You Need | AISC Foundational](https://i.ytimg.com/vi/S0KakHcj_rs/default.jpg)
[Transformer] Attention Is All You Need | AISC Foundational
NLP with Neural Networks & Transformers
Vision Transformer (ViT) - An image is worth 16x16 words | Paper Explained