Hopfield Networks in 2021 - Fireside chat between Sepp Hochreiter and Dmitry Krotov | NeurIPS 2020
Recent ground-breaking research by Sepp Hochreiter, John Hopfield, and Dmitry Krotov reignited the interest in modern Hopfield networks and associative memories. The paper “Hopfield Networks is All You Need” by Sepp Hochreiter’s team introduced a modern continuous modern Hopfield network with exponential storage capacity and very fast convergence. The authors showed that the update rule of the new Hopfield network is equivalent to the attention mechanism of the popular successful Transformer architecture. John Hopfield and Dmitry Krotov then showed in their work on “Large Associative Memory Problem in Neurobiology and Machine Learning” that dense associative memories and modern Hopfield networks can be linked to models with biologically plausible pair-wise interactions between neurons. Together, these recent hallmark studies are important for understanding Transformers, improving attention mechanisms, while providing a link of these advanced architectures to biologically plausible neural networks. We are looking forward to these pioneers of AI exploring the implications of their findings from a theoretical, a biological, and a physics perspectives, and a discussion of future directions of research.
_____________________________________
IARAI - Institute of Advanced Research in Artificial Intelligence
www.iarai.org
Видео Hopfield Networks in 2021 - Fireside chat between Sepp Hochreiter and Dmitry Krotov | NeurIPS 2020 канала IARAI Research
_____________________________________
IARAI - Institute of Advanced Research in Artificial Intelligence
www.iarai.org
Видео Hopfield Networks in 2021 - Fireside chat between Sepp Hochreiter and Dmitry Krotov | NeurIPS 2020 канала IARAI Research
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
Evaluating Machine (Human) Accuracy and Robustness on ImageNet - Ludwig SchmidtA Number Sense as an Emergent Property of the Manipulating Brain - Pietro PeronaEmbedding and Language Modeling for Effective Text Mining - Jiawei HanModern Hopfield Networks - Dr Sepp HochreiterLearned data augmentation in natural language processing - Kyunghyun ChoCDCEO 22: Session I - Invited talk by Vipin KumarCDCEO 22: Session III - Invited talk by Nebojsa JojicTowards General and Robust AI at Scale - Irina RishPerformers & Memory - fireside chat: Sepp Hochreiter, Krzysztof Choromanski & Johannes BrandstetterScience4cast Special Session - Special Prize: Francisco AndradesScience4cast Special Session - 2nd Place: Ngoc TranProtein structure prediction with AlphaFold - Andrew SeniorTraffic4cast Special Session: Part II - NeurIPS 2020Neural diffusion PDEs, differential geometry, and graph neural networks - Michael BronsteinWeather4cast 2021 Special Session - Part 2Machine Learning for Location Based Services - Prof. Dr. Ioannis GiannopoulosCDCEO22: Session II - Invited talk by Nantheera AnantrasirichaiScience4cast Special Session - 3rd Place: Milad AghajohariPatenting AI: Why, What and How? - Dr. Alexander KorenbergThe Importance of Motion Perception in Visual Recognition - Roman Pflugfelder