Topographic VAEs learn Equivariant Capsules (Machine Learning Research Paper Explained)
#tvae #topographic #equivariant
Variational Autoencoders model the latent space as a set of independent Gaussian random variables, which the decoder maps to a data distribution. However, this independence is not always desired, for example when dealing with video sequences, we know that successive frames are heavily correlated. Thus, any latent space dealing with such data should reflect this in its structure. Topographic VAEs are a framework for defining correlation structures among the latent variables and induce equivariance within the resulting model. This paper shows how such correlation structures can be built by correctly arranging higher-level variables, which are themselves independent Gaussians.
OUTLINE:
0:00 - Intro
1:40 - Architecture Overview
6:30 - Comparison to regular VAEs
8:35 - Generative Mechanism Formulation
11:45 - Non-Gaussian Latent Space
17:30 - Topographic Product of Student-t
21:15 - Introducing Temporal Coherence
24:50 - Topographic VAE
27:50 - Experimental Results
31:15 - Conclusion & Comments
Paper: https://arxiv.org/abs/2109.01394
Code: https://github.com/akandykeller/topographicvae
Abstract:
In this work we seek to bridge the concepts of topographic organization and equivariance in neural networks. To accomplish this, we introduce the Topographic VAE: a novel method for efficiently training deep generative models with topographically organized latent variables. We show that such a model indeed learns to organize its activations according to salient characteristics such as digit class, width, and style on MNIST. Furthermore, through topographic organization over time (i.e. temporal coherence), we demonstrate how predefined latent space transformation operators can be encouraged for observed transformed input sequences -- a primitive form of unsupervised learned equivariance. We demonstrate that this model successfully learns sets of approximately equivariant features (i.e. "capsules") directly from sequences and achieves higher likelihood on correspondingly transforming test sequences. Equivariance is verified quantitatively by measuring the approximate commutativity of the inference network and the sequence transformations. Finally, we demonstrate approximate equivariance to complex transformations, expanding upon the capabilities of existing group equivariant neural networks.
Authors: T. Anderson Keller, Max Welling
Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
Parler: https://parler.com/profile/YannicKilcher
LinkedIn: https://www.linkedin.com/in/ykilcher
BiliBili: https://space.bilibili.com/1824646584
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
Видео Topographic VAEs learn Equivariant Capsules (Machine Learning Research Paper Explained) канала Yannic Kilcher
Variational Autoencoders model the latent space as a set of independent Gaussian random variables, which the decoder maps to a data distribution. However, this independence is not always desired, for example when dealing with video sequences, we know that successive frames are heavily correlated. Thus, any latent space dealing with such data should reflect this in its structure. Topographic VAEs are a framework for defining correlation structures among the latent variables and induce equivariance within the resulting model. This paper shows how such correlation structures can be built by correctly arranging higher-level variables, which are themselves independent Gaussians.
OUTLINE:
0:00 - Intro
1:40 - Architecture Overview
6:30 - Comparison to regular VAEs
8:35 - Generative Mechanism Formulation
11:45 - Non-Gaussian Latent Space
17:30 - Topographic Product of Student-t
21:15 - Introducing Temporal Coherence
24:50 - Topographic VAE
27:50 - Experimental Results
31:15 - Conclusion & Comments
Paper: https://arxiv.org/abs/2109.01394
Code: https://github.com/akandykeller/topographicvae
Abstract:
In this work we seek to bridge the concepts of topographic organization and equivariance in neural networks. To accomplish this, we introduce the Topographic VAE: a novel method for efficiently training deep generative models with topographically organized latent variables. We show that such a model indeed learns to organize its activations according to salient characteristics such as digit class, width, and style on MNIST. Furthermore, through topographic organization over time (i.e. temporal coherence), we demonstrate how predefined latent space transformation operators can be encouraged for observed transformed input sequences -- a primitive form of unsupervised learned equivariance. We demonstrate that this model successfully learns sets of approximately equivariant features (i.e. "capsules") directly from sequences and achieves higher likelihood on correspondingly transforming test sequences. Equivariance is verified quantitatively by measuring the approximate commutativity of the inference network and the sequence transformations. Finally, we demonstrate approximate equivariance to complex transformations, expanding upon the capabilities of existing group equivariant neural networks.
Authors: T. Anderson Keller, Max Welling
Links:
TabNine Code Completion (Referral): http://bit.ly/tabnine-yannick
YouTube: https://www.youtube.com/c/yannickilcher
Twitter: https://twitter.com/ykilcher
Discord: https://discord.gg/4H8xxDF
BitChute: https://www.bitchute.com/channel/yannic-kilcher
Minds: https://www.minds.com/ykilcher
Parler: https://parler.com/profile/YannicKilcher
LinkedIn: https://www.linkedin.com/in/ykilcher
BiliBili: https://space.bilibili.com/1824646584
If you want to support me, the best thing to do is to share out the content :)
If you want to support me financially (completely optional and voluntary, but a lot of people have asked for this):
SubscribeStar: https://www.subscribestar.com/yannickilcher
Patreon: https://www.patreon.com/yannickilcher
Bitcoin (BTC): bc1q49lsw3q325tr58ygf8sudx2dqfguclvngvy2cq
Ethereum (ETH): 0x7ad3513E3B8f66799f507Aa7874b1B0eBC7F85e2
Litecoin (LTC): LQW2TRyKYetVC8WjFkhpPhtpbDM4Vw7r9m
Monero (XMR): 4ACL8AGrEo5hAir8A9CeVrW8pEauWvnp1WnSDZxW7tziCDLhZAGsgzhRQABDnFy8yuM9fWJDviJPHKRjV4FWt19CJZN9D4n
Видео Topographic VAEs learn Equivariant Capsules (Machine Learning Research Paper Explained) канала Yannic Kilcher
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
How to combine classic algorithms and AI - Learning with Algorithmic Supervision via RelaxationsHow to build an ML pipeline with TFXFixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence[ML News] DeepMind controls fusion | Yann LeCun's JEPA architecture | US: AI can't copyright its artMachine Learning PhD Survival Guide 2021 | Advice on Topic Selection, Papers, Conferences & more!Weight Standardization (Paper Explained)NVAE: A Deep Hierarchical Variational Autoencoder (Paper Explained)How I Read a Paper: Facebook's DETR (Video Tutorial)Big Bird: Transformers for Longer Sequences (Paper Explained)Big Self-Supervised Models are Strong Semi-Supervised Learners (Paper Explained)OpenAI DALL·E: Creating Images from Text (Blog Post Explained)[ML Olds] Meta Research Supercluster | OpenAI GPT-Instruct | Google LaMDA | Drones fight PigeonsVOS: Learning What You Don't Know by Virtual Outlier Synthesis (Paper Explained)Typical Decoding for Natural Language Generation (Get more human-like outputs from language models!)Rethinking Attention with Performers (Paper Explained)Deep Ensembles: A Loss Landscape Perspective (Paper Explained)Competition-Level Code Generation with AlphaCode (Paper Review)Learning To Classify Images Without Labels (Paper Explained)Why AI is Harder Than We Think (Machine Learning Research Paper Explained)