Cross-Lingual Transfer Learning with DeepMind

Sebastian Ruder, a Research Scientist at DeepMind, joined us at the Conversational AI & NLP Virtual Summit in April 2021:

Research in natural language processing (NLP) has seen striking advances in recent years, mainly driven by large pre-trained language models. However, most of these successes have been achieved in English and a small set of other high-resource languages. In this talk, I will highlight methods that enable us to scale NLP models to more of the world's 7,000 languages, challenges, and promising future directions.

Key takeaways:
- Multilingual models are necessary in order to democratise access to language technology.
- Large language models trained on unlabelled data in many languages learn a surprising amount of cross-lingual information.
- Nevertheless, there are many open challenges such as generalisation to languages with limited data.

Видео Cross-Lingual Transfer Learning with DeepMind канала RE•WORK