Jiayuan Mao - Neuro-Symbolic Frameworks for Visual Concept Learning and Language Acquisition

03/19/2020
Link to slides: https://drive.google.com/open?id=1vqV22iywgws8e6hGU3bbc621cftrixJf

Humans are capable of learning visual concepts by jointly understanding vision and language. Imagine that someone with no prior knowledge of colors is presented with the images of the red and green objects, paired with descriptions. They can easily identify the difference in objects’ visual appearance (in this case, color), and align it to the corresponding words. This intuition motivates the use of image-text pairs to facilitate automated visual concept learning and language acquisition.

In the talk, I will present recent progress on neuro-symbolic models for visual concept learning, reasoning, and language acquisition. These models learn visual concepts and their association with symbolic representations of language and unravel syntactic structures, as well as compositional semantics of sentences, only by looking at images and reading paired natural language texts. No explicit supervision, such as class labels for objects or parse trees, is needed. I will also discuss their extensions to syntactic bootstrapping, metaconcept reasoning, action grounding, and robotic planning.
Jiayuan Mao is a Ph.D. student at MIT, advised by Professors Josh Tenenbaum and Leslie Kaelbling. Mao's research focuses on structured knowledge representations that can be transferred among tasks and inductive biases that improve the learning efficiency and generalization. Representative research topics are concept learning, neuro-symbolic reasoning, scene understanding, language acquisition, and robotic planning.

Видео Jiayuan Mao - Neuro-Symbolic Frameworks for Visual Concept Learning and Language Acquisition канала Vision & Graphics Seminar at MIT