This short tutorial covers the basics of normalizing flows, a technique used in machine learning to build up complex probability distributions by transforming simple ones.
Timestamps:
0:00 - Intro
0:33 - Bijective transformation
1:18 - Change of variables formula
2:08 - Jacobian determinant
4:28 - Generative model likelihood
5:49 - Comparison with VAEs & GANs
6:52 - NICE architecture: triangular Jacobian & coupling layers
9:23 - Scaling matrix
10:26 - Extensions
Papers to check out:
NICE: Non-linear Independent Components Estimation (arxiv.org/abs/1410.8516)
Density estimation using Real NVP (arxiv.org/abs/1605.08803)
Glow: Generative Flow with Invertible 1x1 Convolutions (arxiv.org/abs/1807.03039)
Variational Inference with Normalizing Flows (arxiv.org/abs/1505.05770)
Improving Variational Inference with Inverse Autoregressive Flow (arxiv.org/abs/1606.04934)
Masked Autoregressive Flow for Density Estimation (arxiv.org/abs/1705.07057)
MADE: Masked Autoencoder for Distribution Estimation (arxiv.org/abs/1502.03509)
Discrete Flows: Invertible Generative Models of Discrete Data (arxiv.org/abs/1905.10347)
Earlier work on flows:
A family of non-parametric density estimation algorithms (math.nyu.edu/faculty/tabak/pu...)
Additional reading:
deepgenerativemodels.github.i...
blog.evjang.com/2018/01/nf1.html
lilianweng.github.io/lil-log/...
akosiorek.github.io/ml/2018/04...
Special thanks to Alex Beatson, Geoffrey Roeder, Yaniv Ovadia, Sachin Ravi, and Ryan Adams for helpful feedback on this video.
Video style inspired by 3Blue1Brown
Music: Trinkets by Vincent Rubinetti
Links:
KZfaq: / ariseffai
Twitter: / ari_seff
Homepage: www.ariseff.com
If you'd like to help support the channel (completely optional), you can donate a cup of coffee via the following:
Venmo: venmo.com/ariseff
PayPal: www.paypal.me/ariseff