Transformers vs Recurrent Neural Networks (RNN)!
Course link: https://www.coursera.org/learn/attention-models-in-nlp/lecture/glNgT/transformers-vs-rnns
Using an RNN, you have to take sequential steps to encode your input, and you start from the beginning of your input making computations at every step until you reach the end. At that point, you decode the information following a similar sequential procedure. As you can see here, you have to go through every word in your inputs starting with the first word followed by the second word, one after another. In sequential matcher in order to start the translation, that is done in a sequential way too. For that reason, there is not much room for parallel computations here. The more words you have in the input sequence, the more time it will take to process that sentence. Take a look at a more general sequence to sequence architecture.In this case, to propagate information from your first word to the last output, you have to go through T sequential steps.
Видео Transformers vs Recurrent Neural Networks (RNN)! канала Machine Learning TV
Using an RNN, you have to take sequential steps to encode your input, and you start from the beginning of your input making computations at every step until you reach the end. At that point, you decode the information following a similar sequential procedure. As you can see here, you have to go through every word in your inputs starting with the first word followed by the second word, one after another. In sequential matcher in order to start the translation, that is done in a sequential way too. For that reason, there is not much room for parallel computations here. The more words you have in the input sequence, the more time it will take to process that sentence. Take a look at a more general sequence to sequence architecture.In this case, to propagate information from your first word to the last output, you have to go through T sequential steps.
Видео Transformers vs Recurrent Neural Networks (RNN)! канала Machine Learning TV
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
Limitations of the ChatGPT and LLMs - Part 3
Understanding ChatGPT and LLMs from Scratch - Part 2
Understanding ChatGPT and LLMs from Scratch - Part 1
Understanding BERT Embeddings and How to Generate them in SageMaker
Understanding Coordinate Descent
Bootstrap and Monte Carlo Methods
Maximum Likelihood as Minimizing KL Divergence
Understanding The Shapley Value
Kalman Filter - Part 2
Kalman Filter - Part 1
Recurrent Neural Networks (RNNs) and Vanishing Gradients
Language Model Evaluation and Perplexity
Common Patterns in Time Series: Seasonality, Trend and Autocorrelation
Limitations of Graph Neural Networks (Stanford University)
Understanding Metropolis-Hastings algorithm
Learning to learn: An Introduction to Meta Learning
Page Ranking: Web as a Graph (Stanford University 2019)
Deep Graph Generative Models (Stanford University - 2019)
Graph Node Embedding Algorithms (Stanford - Fall 2019)
Graph Representation Learning (Stanford university)