#54 Prof. GARY MARCUS + Prof. LUIS LAMB - Neurosymbolic models

Professor Gary Marcus is a scientist, best-selling author, and entrepreneur. He is Founder and CEO of Robust.AI, and was Founder and CEO of Geometric Intelligence, a machine learning company acquired by Uber in 2016. Gary said in his recent next decade paper that — without us, or other creatures like us, the world would continue to exist, but it would not be described, distilled, or understood. Human lives are filled with abstraction and causal description. This is so powerful. Francois Chollet the other week said that intelligence is literally sensitivity to abstract analogies, and that is all there is to it. It's almost as if one of the most important features of intelligence is to be able to abstract knowledge, this drives the generalisation which will allow you to mine previous experience to make sense of many future novel situations.

Also joining us today is Professor Luis Lamb — Secretary of Innovation for Science and Technology of the State of Rio Grande do Sul, Brazil. His Research Interests are Machine Learning and Reasoning, Neuro-Symbolic Computing, Logic in Computation and Artificial Intelligence, Cognitive and Neural Computation and also AI Ethics and Social Computing. Luis released his new paper Neurosymbolic AI: the third wave at the end of last year. It beautifully articulated the key ingredients needed in the next generation of AI systems, integrating type 1 and type 2 approaches to AI and it summarises all the of the achievements of the last 20 years of research.

We cover a lot of ground in today's show. Explaining the limitations of deep learning, Rich Sutton's the bitter lesson and "reward is enough", and the semantic foundation which is required for us to build robust AI.

Pod: https://anchor.fm/machinelearningstreettalk/episodes/54-Gary-Marcus-and-Luis-Lamb---Neurosymbolic-models-e125495

Tim Epic Intro [00:00:00]
Main Intro [00:38:05]
Gary introduces the field [00:42:12]
Luis introduces his thoughts on Neurosymbolic methods [00:47:56]
On the history of achieving a logical foundation and mathematical foundation for semantics [00:54:12]
Will emulating discrete reasoning break optimizability?
Buzzwords without basis [01:04:34]
We have known for decades about the statistical regularities in language [01:07:02]
Intension vs extension [01:09:14]
Easy to demand abstraction, but what is a workable definition? [01:13:33]
Abstraction is a "terrorist attack on neural networks" [01:20:38]
To succeed we need both, we are the moderates [01:30:14]
What would the future world look like with better semantics? [01:31:32]
Promising current approaches to discrete reasoning systems [01:39:58]
The challenge of machine knowledge acquisition [01:47:32]
Prof. Lamb's more on relational learning [01:53:06]
The role of vector embeddings and neural symbolics [02:02:30]
Humans seem both good and bad at reasoning, what's going on? [02:09:06]
Is reasoning a first-class citizen in the human brain? [02:15:06]
Does reasoning happen on the same substrate as system 1? [02:17:08]

GM papers:

The Next Decade in AI
https://arxiv.org/abs/2002.06177

Innateness, AlphaZero, and Artificial Intelligence
https://arxiv.org/abs/1801.05667

Deep Learning: A Critical Appraisal
https://arxiv.org/abs/1801.00631

Rule learning by seven-month-old infants
https://www.researchgate.net/publication/13415195_Rule_learning_by_seven-month-old_infants

Rethinking Eliminative Connectionism
https://nyuscholars.nyu.edu/en/publications/rethinking-eliminative-connectionism

GM YB Debate
The Best Way Forward For AI
https://montrealartificialintelligence.com/aidebate/

GM:

Rebooting AI
https://www.amazon.com/Rebooting-AI-Building-Artificial-Intelligence-ebook/dp/B07MYLGQLB

Kluge: The Haphazard Evolution of the Human Mind
https://www.amazon.com/Kluge-Haphazard-Evolution-Human-Mind-ebook/dp/B003JTHWQ4

The Birth of The Mind
https://www.amazon.com/Birth-Mind-Creates-Complexities-Thought/dp/0465044069

The Algebraic Mind
https://www.amazon.com/Algebraic-Mind-Integrating-Connectionism-Development/dp/0262632683

LL:

Neurosymbolic AI: The 3rd Wave
https://arxiv.org/pdf/2012.05876.pdf

Understanding Boolean Function Learnability on Deep Neural Networks
https://arxiv.org/pdf/2009.05908.pdf

Graph Neural Networks Meet Neural-Symbolic Computing
https://arxiv.org/abs/2003.00330

Discrete and Continuous Deep Residual Learning Over Graphs
https://arxiv.org/pdf/1911.09554.pdf

Learning to Solve NP-Complete Problems
https://arxiv.org/abs/1809.02721

Neural-symbolic Computing
https://arxiv.org/pdf/1905.06088.pdf

Neural-symbolic learning and reasoning
https://arxiv.org/abs/1711.03902

Neural-symbolic learning and reasoning
https://openaccess.city.ac.uk/id/eprint/11838/

LL books:

Neural-Symbolic Cognitive Reasoning
https://www.amazon.com/Neural-Symbolic-Cognitive-Reasoning-Technologies/dp/3642092292

A Uniform Presentation of Non-Classical Logics
https://www.amazon.com/Compiled-Labelled-Deductive-Systems-Non-Classical/dp/0863802966

Видео #54 Prof. GARY MARCUS + Prof. LUIS LAMB - Neurosymbolic models канала Machine Learning Street Talk