Week 7 – Lecture: Energy based models and self-supervised learning

Course website: http://bit.ly/pDL-home
Playlist: http://bit.ly/pDL-YouTube
Speaker: Yann LeCun
Week 7: http://bit.ly/pDL-en-07

0:00:00 – Week 7 – Lecture

LECTURE Part A: http://bit.ly/pDL-en-07-1
We introduced the concept of the energy-based models and the intention for different approaches other than feed-forward networks. To solve the difficulty of the inference in EBM, latent variables are used to provide auxiliary information and enable multiple possible predictions. Finally, the EBM can generalize to probabilistic model with more flexible scoring functions.
0:01:04 – Energy-based model concept
0:15:04 – Latent-variable EBM: inference
0:28:23 – EBM vs. probabilistic models

LECTURE Part B: http://bit.ly/pDL-en-07-2
We discussed self-supervised learning, introduced how to train an Energy-based models, discussed Latent Variable EBM, specifically with an explained K-means example. We also introduced Contrastive Methods, explained a denoising autoencoder with a topographic map, the training process, and how it can be used, followed by an introduction to BERT. Finally, we talked about Contrastive Divergence, also explained using a topographic map.
0:44:43 – Self-supervised learning
1:05:57 – Training an Energy-Based Model
1:19:27 – Latent Variable EBM, K-means example, Contrastive Methods

Видео Week 7 – Lecture: Energy based models and self-supervised learning канала Alfredo Canziani