What are Normalizing Flows?

This short tutorial covers the basics of normalizing flows, a technique used in machine learning to build up complex probability distributions by transforming simple ones.

Papers to check out:
NICE: Non-linear Independent Components Estimation (https://arxiv.org/abs/1410.8516)
Density estimation using Real NVP (https://arxiv.org/abs/1605.08803)
Glow: Generative Flow with Invertible 1x1 Convolutions (https://arxiv.org/abs/1807.03039)
Variational Inference with Normalizing Flows (https://arxiv.org/abs/1505.05770)
Improving Variational Inference with Inverse Autoregressive Flow (https://arxiv.org/abs/1606.04934)
Masked Autoregressive Flow for Density Estimation (https://arxiv.org/abs/1705.07057)
MADE: Masked Autoencoder for Distribution Estimation (https://arxiv.org/abs/1502.03509)
Discrete Flows: Invertible Generative Models of Discrete Data (https://arxiv.org/abs/1905.10347)

Earlier work on flows:
A family of non-parametric density estimation algorithms (https://math.nyu.edu/faculty/tabak/publications/Tabak-Turner.pdf)

Additional reading:
https://deepgenerativemodels.github.io/notes/flow/
https://blog.evjang.com/2018/01/nf1.html
https://lilianweng.github.io/lil-log/2018/10/13/flow-based-deep-generative-models.html
http://akosiorek.github.io/ml/2018/04/03/norm_flows.html

Special thanks to Alex Beatson, Geoffrey Roeder, Yaniv Ovadia, Sachin Ravi, and Ryan Adams for helpful feedback on this video.

Video style inspired by 3Blue1Brown

Music: Trinkets by Vincent Rubinetti

Видео What are Normalizing Flows? канала Ari Seff