The Map Interpretation of Attention — Mat Kelcey

Mat Kelcey "The map interpretation of attention" Attention mechanisms are now a fundamental building block for neural networks in natural language processing. In this talk we'll go over a old problem in NLP and how attention originally solved it; all using the interpretation of how attention acts as a form of soft map lookup. We'll then show how modifications to this form the basis of the Transformer architecture.
Recorded from the Machine Learning and AI Meetup Melbourne August 2020 event

Видео The Map Interpretation of Attention — Mat Kelcey канала Machine Learning and AI Meetup