Transformer Model (1/2): Attention Layers
Next Video: https://youtu.be/J4H6A4-dvhE
The Transformer models are state-of-the-art language models. They are based on attention and dense layer without RNN. Instead of studying every module of Transformer, let us try to build a Transformer model from scratch. In this lecture, we eliminate RNNs while keeping attentions. We will get an attention layer and a self-attention layer. In the next lecture, we use attention, self-attention, and dense layers to build a deep neural network which is known as Transformer.
Slides: https://github.com/wangshusen/DeepLearning
Reference:
Vaswani et al. Attention Is All You Need. In NIPS, 2017.
Видео Transformer Model (1/2): Attention Layers канала Shusen Wang
The Transformer models are state-of-the-art language models. They are based on attention and dense layer without RNN. Instead of studying every module of Transformer, let us try to build a Transformer model from scratch. In this lecture, we eliminate RNNs while keeping attentions. We will get an attention layer and a self-attention layer. In the next lecture, we use attention, self-attention, and dense layers to build a deep neural network which is known as Transformer.
Slides: https://github.com/wangshusen/DeepLearning
Reference:
Vaswani et al. Attention Is All You Need. In NIPS, 2017.
Видео Transformer Model (1/2): Attention Layers канала Shusen Wang
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
Transformer Model (2/2): Build a Deep Neural Network (1.25x speed recommended)
Few-Shot Learning (1/3): Basic Concepts
Vision Transformer for Image Classification
Self-Attenion for RNN (1.25x speed recommended)
The secrets of learning a new language | Lýdia Machová
Rasa Algorithm Whiteboard - Transformers & Attention 1: Self Attention
Visual Guide to Transformer Neural Networks - (Episode 2) Multi-Head & Self-Attention
Sequence to Sequence Learning with Encoder-Decoder Neural Network Models by Dr. Ananth Sankar![[딥러닝 기계 번역] Transformer: Attention Is All You Need (꼼꼼한 딥러닝 논문 리뷰와 코드 실습)](https://i.ytimg.com/vi/AA621UofTUA/default.jpg)
[딥러닝 기계 번역] Transformer: Attention Is All You Need (꼼꼼한 딥러닝 논문 리뷰와 코드 실습)
How does a Transformer work - Working Principle electrical engineering
Attention Mechanism | Deep Learning
Will Transformers Replace CNNs in Computer Vision? + NVIDIA GTC Giveaway
BERT for pretraining Transformers
RAM Explained - Random Access Memory
Illustrated Guide to LSTM's and GRU's: A step by step explanation
Transformer Neural Networks - EXPLAINED! (Attention is all you need)
Recurrent Neural Networks | RNN LSTM Tutorial | Why use RNN | On Whiteboard | Compare ANN, CNN, RNN
Rasa Algorithm Whiteboard - Transformers & Attention 3: Multi Head Attention
The Narrated Transformer Language Model
Attention for RNN Seq2Seq Models (1.25x speed recommended)