The space of functions computed by deep-learning networks, David Saad | LMS/IMA Joint Meeting 2023
Recent engineering achievements of deep-learning machines have both impressed and intrigued the scientific community due to our limited theoretical understanding of the underlying reasons for their success. I will briefly review some of the challenges to be addressed and then focus on properties of the function space of different types of deep-learning machines, based on the generating functional analysis. This approach facilitates studying the number of solution networks of a given error around a reference multi-layer network. Exploring the function landscape of densely-connected networks, we uncover a general layer-by-layer learning behaviour, while the study of sparsely-connected networks indicates the advantage in having more layers for increasing generalization ability in such models. This framework accommodates other network architectures and computing elements, including networks with correlated weights, convolutional networks and discretised variables. A similar approach also facilitates studying the distribution of Boolean functions computed by recurrent and layer-dependent architectures, which we find to be the same. Depending on the initial conditions and computing elements used, we characterize the space of functions computed at the large depth limit and show that the macroscopic entropy of Boolean functions is either monotonically increasing or decreasing with the growing depth.
Part of the LMS/IMA Joint Meeting on 'The Mathematical Foundations' of AI, which took place on Friday 13 October 2023 at De Morgan House, London and online via Zoom.
==========
The London Mathematical Society has, since 1865, been the UK's learned society for the advancement, dissemination and promotion of mathematical knowledge. Our mission is to advance mathematics through our members and the broader scientific community worldwide.
For further information:
► Website: https://www.lms.ac.uk
► Events: https://www.lms.ac.uk/events
► Grants and Prizes: https://www.lms.ac.uk/grants-prizes
► Publications: https://www.lms.ac.uk/publications
► Membership: https://www.lms.ac.uk/membership
Follow us:
► Twitter: https://twitter.com/LondMathSoc
► Facebook: https://www.facebook.com/londonmathematicalsociety
► LinkedIn: https://www.linkedin.com/company/the-london-mathematical-society/
► Youtube: @LondonMathematicalSociety
Видео The space of functions computed by deep-learning networks, David Saad | LMS/IMA Joint Meeting 2023 канала London Mathematical Society
Part of the LMS/IMA Joint Meeting on 'The Mathematical Foundations' of AI, which took place on Friday 13 October 2023 at De Morgan House, London and online via Zoom.
==========
The London Mathematical Society has, since 1865, been the UK's learned society for the advancement, dissemination and promotion of mathematical knowledge. Our mission is to advance mathematics through our members and the broader scientific community worldwide.
For further information:
► Website: https://www.lms.ac.uk
► Events: https://www.lms.ac.uk/events
► Grants and Prizes: https://www.lms.ac.uk/grants-prizes
► Publications: https://www.lms.ac.uk/publications
► Membership: https://www.lms.ac.uk/membership
Follow us:
► Twitter: https://twitter.com/LondMathSoc
► Facebook: https://www.facebook.com/londonmathematicalsociety
► LinkedIn: https://www.linkedin.com/company/the-london-mathematical-society/
► Youtube: @LondonMathematicalSociety
Видео The space of functions computed by deep-learning networks, David Saad | LMS/IMA Joint Meeting 2023 канала London Mathematical Society
Показать
Комментарии отсутствуют
Информация о видео
31 октября 2023 г. 16:59:54
00:55:31
Другие видео канала
How Maths Untangles Knotty DNA Questions, Dorothy Buck | LMS Popular Lectures 2010Mathematics in the Courtroom, Ray Hill | LMS Popular Lectures 2013LMS-Bath Symposium 2020, Rates of Convergence for Cheeger Cuts on Point Clouds, Matthew ThorpeUtilising Shape in Data, Ulrike Tillmann FRS | LMSOn generalisation and learning, Benjamin Guedj | LMS Computer Science ColloquiumLMS-Bath Symposium 2020, Public Lecture, Carola-Bibiane SchönliebNet of Conics, Vanya Cheltsov | LMS Summer School (2/5)The Mathematics of Processing Digital Images, Joan Lasenby | LMS Popular Lectures 2015LMS-Bath Symposium 2020, Machine learning from a continuous viewpoint, Weinan ECombinatorics of Young tableaux and symmetric groups, Sarah Whitehouse | LMS Summer School (2/4)The Journey of Female African Mathematicians, Angela Tabiri | LMS BHoM 2020Risky Business, Jen Rogers | LMS Popular Lectures 2018Soap Bubbles and Minimal Surfaces, Spencer Becker-Kahn | LMS BHoMLMS-Bath Symposium 2020, A PDE Interpretation of Prediction with Expert Advice, Nadia DrenskaProfessor Noga Alon speaking at the LMS/EMS Anniversary Mathematical Weekend 2015Mary Cartwright Lecture 2021, Analyticity in the Sky with (causal) Diamonds, Claudia de RhamVerification of control software for robots that learn, Ana Cavalcanti | LMS CS Colloquium 2023Bayesian inference with data-driven image priors: theory, methods, and algorithms, Marcelo PereyraThe Riddle of Robustness, Flexibility, and Trustworthiness of Data-driven AI, Ivan TyukinTropical geometry: Algebra, geometry and combinatorics, Felipe Rincon | LMS Summer School