Weaving together machine learning, theoretical physics, and neuroscience through mathematics
11/4/2020 Colloquium
Speaker: Surya Ganguli (Stanford)
Title: Weaving together machine learning, theoretical physics, and neuroscience through mathematics
Abstract: An exciting area of intellectual activity in this century may well revolve around a synthesis of machine learning, theoretical physics, and neuroscience. The unification of these fields will likely enable us to exploit the power of complex systems analysis, developed in theoretical physics and applied mathematics, to elucidate the design principles governing neural systems, both biological and artificial, and deploy these principles to develop better algorithms in machine learning. We will give several vignettes in this direction, including: (1) determining the best optimization problem to solve in order to perform regression in high dimensions; (2) finding exact solutions to the dynamics of generalization error in deep linear networks; (3) developing interpretable machine learning to derive and understand state of the art models of the retina; (4) analyzing and explaining the origins of hexagonal firing patterns in recurrent neural networks trained to path-integrate; (5) delineating fundamental theoretical limits on the energy, speed and accuracy with which non-equilibrium sensors can detect signals
Selected References:
M. Advani and S. Ganguli, Statistical mechanics of optimal convex inference in high dimensions, Physical Review X, 6, 031034, 2016.
M. Advani and S. Ganguli, An equivalence between high dimensional Bayes optimal inference and M-estimation, NeurIPS, 2016.
A.K. Lampinen and S. Ganguli, An analytic theory of generalization dynamics and transfer learning in deep linear networks, International Conference on Learning Representations (ICLR), 2019.
H. Tanaka, A. Nayebi, N. Maheswaranathan, L.M. McIntosh, S. Baccus, S. Ganguli, From deep learning to mechanistic understanding in neuroscience: the structure of retinal prediction, NeurIPS 2019.
S. Deny, J. Lindsey, S. Ganguli, S. Ocko, The emergence of multiple retinal cell types through efficient coding of natural movies, Neural Information Processing Systems (NeurIPS) 2018.
B. Sorscher, G. Mel, S. Ganguli, S. Ocko, A unified theory for the origin of grid cells through the lens of pattern formation, NeurIPS 2019.
Y. Bahri, J. Kadmon, J. Pennington, S. Schoenholz, J. Sohl-Dickstein, and S. Ganguli, Statistical mechanics of deep learning, Annual Reviews of Condensed Matter Physics, 2020.
S.E. Harvey, S. Lahiri, and S. Ganguli, A universal energy accuracy tradeoff in nonequilibrium cellular sensing, https://arxiv.org/abs/2002.10567
Видео Weaving together machine learning, theoretical physics, and neuroscience through mathematics канала Harvard CMSA
Speaker: Surya Ganguli (Stanford)
Title: Weaving together machine learning, theoretical physics, and neuroscience through mathematics
Abstract: An exciting area of intellectual activity in this century may well revolve around a synthesis of machine learning, theoretical physics, and neuroscience. The unification of these fields will likely enable us to exploit the power of complex systems analysis, developed in theoretical physics and applied mathematics, to elucidate the design principles governing neural systems, both biological and artificial, and deploy these principles to develop better algorithms in machine learning. We will give several vignettes in this direction, including: (1) determining the best optimization problem to solve in order to perform regression in high dimensions; (2) finding exact solutions to the dynamics of generalization error in deep linear networks; (3) developing interpretable machine learning to derive and understand state of the art models of the retina; (4) analyzing and explaining the origins of hexagonal firing patterns in recurrent neural networks trained to path-integrate; (5) delineating fundamental theoretical limits on the energy, speed and accuracy with which non-equilibrium sensors can detect signals
Selected References:
M. Advani and S. Ganguli, Statistical mechanics of optimal convex inference in high dimensions, Physical Review X, 6, 031034, 2016.
M. Advani and S. Ganguli, An equivalence between high dimensional Bayes optimal inference and M-estimation, NeurIPS, 2016.
A.K. Lampinen and S. Ganguli, An analytic theory of generalization dynamics and transfer learning in deep linear networks, International Conference on Learning Representations (ICLR), 2019.
H. Tanaka, A. Nayebi, N. Maheswaranathan, L.M. McIntosh, S. Baccus, S. Ganguli, From deep learning to mechanistic understanding in neuroscience: the structure of retinal prediction, NeurIPS 2019.
S. Deny, J. Lindsey, S. Ganguli, S. Ocko, The emergence of multiple retinal cell types through efficient coding of natural movies, Neural Information Processing Systems (NeurIPS) 2018.
B. Sorscher, G. Mel, S. Ganguli, S. Ocko, A unified theory for the origin of grid cells through the lens of pattern formation, NeurIPS 2019.
Y. Bahri, J. Kadmon, J. Pennington, S. Schoenholz, J. Sohl-Dickstein, and S. Ganguli, Statistical mechanics of deep learning, Annual Reviews of Condensed Matter Physics, 2020.
S.E. Harvey, S. Lahiri, and S. Ganguli, A universal energy accuracy tradeoff in nonequilibrium cellular sensing, https://arxiv.org/abs/2002.10567
Видео Weaving together machine learning, theoretical physics, and neuroscience through mathematics канала Harvard CMSA
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
Arthur Jaffe | Is relativity compatible with quantum theory?Ian Goodfellow: Adversarial Machine Learning (ICLR 2019 invited talk)The Neuroscience of Memory - Eleanor MaguireNigel Hitchin | Michael Atiyah: Geometry and PhysicsTesla Valve | The complete physicsKevin Buzzard | Teaching proofs to computersNeural networks and the brain: from the retina to semantic cognition - Surya GanguliTheoretical physics: insider's tricksLINEN - Making Linen Fabric from Flax Seed - Demonstration Of How Linen Is MadeEdward Witten | Knot Invariants From Gauge Theory in Three, Four, and Five DimensionsMachine Learning in NeuroscienceAndrei Okounkov | Classical and quantum integrable systems in enumerative geometryAlain Connes | Noncommutative Geometry, the Spectral AspectJonathan Heckman | Top Down Approach to Quantum FieldsString theory - Brian GreeneFree Energy Principle — Karl FristonYoshua Bengio | From System 1 Deep Learning to System 2 Deep Learning | NeurIPS 2019David Morrison: The Remarriage of Pure Mathematics and Theoretical Physics [2015]For the Love of Physics - Walter Lewin - May 16, 2011Mike Jordan, "Computational thinking, inferential thinking and Big Data"