Rockwood Memorial Lecture 2021 - Yoshua Bengio, Ph.D., Mila, Université de Montréal (7/26/21)

From Conscious Processing to System 2 Deep Learning

Abstract: Humans are very good at out-of-distribution generalization (at least compared to current AI systems) and it would be good to understand some of the inductive biases they may exploit and test these theories by evaluating how they can be translated into successful ML architectures, training frameworks and experiments. Natural language and experimental results in cognitive science and neuroscience provide a wealth of clues about the system 2 part of how humans understand the world and reason about it. In this talk, I will discuss several of these hypothesized inductive biases, abstracted away from the neural substrate, many of which exploit notions in causality and connect the discovery of abstractions in representation learning (the perception and interpretation part) and in reinforcement learning (the abstract actions). Systematic generalization is hypothesized to arise from an efficient factorization of knowledge into recomposable pieces corresponding to reusable factors (in a directed factor graph formulation). Sparsity of the causal graph and locality of interventions -- which can be observed in the structure of sentences -- have the potential to considerably reduce the computational complexity of both inference (including planning) and learning, which may be a reason for which evolution may have incorporated this "consciousness" prior. Although this talk will rest on a series of recent papers on these topics (e.g., on learning causal and/or modular structure with deep learning), much of it will be forward-facing and suggest open research questions in the hope of stimulating novel investigations and collaborations.

Bio: Recognized worldwide as one of the leading experts in artificial intelligence, Yoshua Bengio is most known for his pioneering work in deep learning, earning him the 2018 A.M. Turing Award, “the Nobel Prize of Computing,” with Geoffrey Hinton and Yann LeCun. He is a Full Professor at Université de Montréal, and the Founder and Scientific Director of Mila – Quebec AI Institute. He co-directs the CIFAR Learning in Machines & Brains program as Senior Fellow and acts as Scientific Director of IVADO. In 2019, he was awarded the prestigious Killam Prize and in 2021, became the second most cited computer scientist in the world. He is a Fellow of both the Royal Society of London and Canada and Officer of the Order of Canada. Concerned about the social impact of AI and the objective that AI benefits all, he actively contributed to the Montreal Declaration for the Responsible Development of Artificial Intelligence.

Видео Rockwood Memorial Lecture 2021 - Yoshua Bengio, Ph.D., Mila, Université de Montréal (7/26/21) канала Institute for Neural Computation