John Wright - Deep Networks and the Multiple Manifold Problem
Prof. John Wright of Columbia University speaking in the UW Data-driven methods in science and engineering seminar on October 7, 2022.
Sign up for notifications of future talks: https://mailman11.u.washington.edu/mailman/listinfo/datadriven-seminar
Abstract: Data in science and engineering often exhibit nonlinear, low-dimensional structure, due to the physical laws that govern data generation. In this talk, we study how deep neural networks interact with structured data:
+ When can we guarantee to fit and generalize?
+ How do the resources (depth, width, data) required depend on the complexity of the data?
+ How can we leverage physical prior knowledge to reduce these resource requirements?
Our main mathematical result is a guarantee of generalization for a model classification problem involving data on low-dimensional manifolds — we prove that for networks of polynomial width, with polynomially many samples, randomly initialized gradient descent rapidly converges to a solution which correctly labels *every* point on the two manifolds. To our knowledge this is the first such result for deep networks on data which are not linearly separable. We highlight intuitions about the roles of depth, width, sample complexity, and the geometry of feature representations, which may be useful in analyzing other problems involving low-dimensional structure (e.g., model discovery). We illustrate these ideas through applied problems in astrophysics and computer vision. In these settings, we further suggest how incorporating physical prior knowledge can reduce the resources (architecture, data) required for learning, leading to more efficient and interpretable learning architectures.
Видео John Wright - Deep Networks and the Multiple Manifold Problem канала Physics Informed Machine Learning
Sign up for notifications of future talks: https://mailman11.u.washington.edu/mailman/listinfo/datadriven-seminar
Abstract: Data in science and engineering often exhibit nonlinear, low-dimensional structure, due to the physical laws that govern data generation. In this talk, we study how deep neural networks interact with structured data:
+ When can we guarantee to fit and generalize?
+ How do the resources (depth, width, data) required depend on the complexity of the data?
+ How can we leverage physical prior knowledge to reduce these resource requirements?
Our main mathematical result is a guarantee of generalization for a model classification problem involving data on low-dimensional manifolds — we prove that for networks of polynomial width, with polynomially many samples, randomly initialized gradient descent rapidly converges to a solution which correctly labels *every* point on the two manifolds. To our knowledge this is the first such result for deep networks on data which are not linearly separable. We highlight intuitions about the roles of depth, width, sample complexity, and the geometry of feature representations, which may be useful in analyzing other problems involving low-dimensional structure (e.g., model discovery). We illustrate these ideas through applied problems in astrophysics and computer vision. In these settings, we further suggest how incorporating physical prior knowledge can reduce the resources (architecture, data) required for learning, leading to more efficient and interpretable learning architectures.
Видео John Wright - Deep Networks and the Multiple Manifold Problem канала Physics Informed Machine Learning
Показать
Комментарии отсутствуют
Информация о видео
21 октября 2022 г. 22:01:28
01:08:19
Другие видео канала
![Benjamin Peherstorfer - Physics-based machine learning for quickly simulating transport-dominated...](https://i.ytimg.com/vi/9Cxgu9TuY2U/default.jpg)
![Aditya Nair/Floris van Breugel - Leveraging Machine Learning for interaction-based models and contro](https://i.ytimg.com/vi/TFGMAtlqaJQ/default.jpg)
![Melanie Weber - Exploiting Geometric Structure in Machine Learning and Optimization](https://i.ytimg.com/vi/EOdnedBuJN0/default.jpg)
![Na Li - Scalable and Safe Learning-based Decision-Making in Dynamical Systems](https://i.ytimg.com/vi/5pHjryDp6lw/default.jpg)
![Alex Gorodetsky - Scalable Learning of Dynamical Systems](https://i.ytimg.com/vi/nrd4lw2qYMk/default.jpg)
![Rose Yu - Incorporating Symmetry for Learning Spatiotemporal Dynamics](https://i.ytimg.com/vi/h7TfFssBFEs/default.jpg)
![Hod Lipson - The AI Scientist Automating Discovery, From Cognitive Robotics to Computational Biology](https://i.ytimg.com/vi/0mKguCNHlEo/default.jpg)
![Steve Brunton - Discovering interpretable and generalizable dynamical systems from data](https://i.ytimg.com/vi/UeyRETIwcYw/default.jpg)
![Jane Bae - Wall-models of turbulent flows via scientific multi-agent reinforcement learning](https://i.ytimg.com/vi/dDS1TUx70J0/default.jpg)
![Kathleen Champion - Data-driven discovery of coordinates and governing equations](https://i.ytimg.com/vi/Z-l7G8zq8I0/default.jpg)
![Kevin Carlberg - Breaking Komolgorov-Width Barriers using Deep Learning](https://i.ytimg.com/vi/kxEiJLNqW_4/default.jpg)
![David Bortz - The Surprising Robustness and Computational Efficiency of Weak Form System Identif...](https://i.ytimg.com/vi/LLDFkOoxDBo/default.jpg)
![Dominique Zosso - Graph-Based Geometric Data Analysis](https://i.ytimg.com/vi/kjZG2ryLmkU/default.jpg)
![Laure Zanna - Data-driven turbulence closures for ocean and climate models: advances and challenges](https://i.ytimg.com/vi/J48yprEYNHk/default.jpg)
![Benjamin Peherstorfer - Data generation for learning reduced models with operator inference](https://i.ytimg.com/vi/odZmGIJYZ14/default.jpg)
![Andrea Bertozzi - Total Variation Minimization on Graphs for Semisupervised and Unsupervised ML](https://i.ytimg.com/vi/EOlG5SGqfL4/default.jpg)
![Andrew Stuart - Supervised Learning For Operators](https://i.ytimg.com/vi/tASot9j7-Cc/default.jpg)
![Krithika Manohar - Optimal Sensors for Empowering AI](https://i.ytimg.com/vi/pDxdjhRU7ak/default.jpg)
![Jörn Dunkel - Symmetry-informed model inference for active matter](https://i.ytimg.com/vi/A53PciowtBE/default.jpg)
![Aleksandr Aravkin - Algorithms for Nonsmooth, Nonconvex Problems in Data-Driven Discovery](https://i.ytimg.com/vi/UnIcBCPO1zg/default.jpg)