Transformer Neural Networks - EXPLAINED! (Attention is all you need)

Please subscribe to keep me alive: https://www.youtube.com/c/CodeEmporium?sub_confirmation=1

SPONSOR
Kite is a free AI-powered coding assistant that will help you code faster and smarter. The Kite plugin integrates with all the top editors and IDEs to give you smart completions and documentation while you’re typing. I've been using Kite. Love it! https://www.kite.com/get-kite/?utm_medium=referral&utm_source=youtube&utm_campaign=codeemporium&utm_content=description-only

REFERENCES
[1] The main Paper: https://arxiv.org/abs/1706.03762
[2] Tensor2Tensor has some code with a tutorial: https://www.tensorflow.org/tutorials/text/transformer
[3] Transformer very intuitively explained - Amazing: http://jalammar.github.io/illustrated-transformer/
[4] Medium Blog on intuitive explanation: https://medium.com/inside-machine-learning/what-is-a-transformer-d07dd1fbec04
[5] Pretrained word embeddings: https://nlp.stanford.edu/projects/glove/
[6] Intuitive explanation of Layer normalization: https://mlexplained.com/2018/11/30/an-overview-of-normalization-methods-in-deep-learning/
[7] Paper that gives even better results than transformers (Pervasive Attention): https://arxiv.org/abs/1808.03867
[8] BERT uses transformers to pretrain neural nets for common NLP tasks. : https://ai.googleblog.com/2018/11/open-sourcing-bert-state-of-art-pre.html
[9] Stanford Lecture on RNN: http://cs231n.stanford.edu/slides/2018/cs231n_2018_lecture10.pdf
[10] Colah’s Blog: https://colah.github.io/posts/2015-08-Understanding-LSTMs/
[11] Wiki for timeseries of events: https://en.wikipedia.org/wiki/Transformer_(machine_learning_model)

Видео Transformer Neural Networks - EXPLAINED! (Attention is all you need) канала CodeEmporium