Deep Networks Are Kernel Machines, Pedro Domingos
Pedro Domingos, Professor of Computer Science at the University of Washington
Deep learning's successes are often attributed to its ability to automatically discover new representations of the data, rather than relying on handcrafted features like other learning methods. In this talk, however, Pedro Domingos will show that deep networks learned by the standard gradient descent algorithm are in fact mathematically approximately equivalent to kernel machines, a learning method that simply memorizes the data and uses it directly for prediction via a similarity function (the kernel). This greatly enhances the interpretability of deep network weights, by elucidating that they are effectively a superposition of the training examples. The network architecture incorporates knowledge of the target function into the kernel. The talk will include a discussion of both the main ideas behind this result and some of its more startling consequences for deep learning, kernel machines, and machine learning at large.
For his 2020 paper behind this talk, see: "Every Model Learned by Gradient Descent Is Approximately a Kernel Machine" at https://arxiv.org/abs/2012.00152
Pedro Domingos is a professor of computer science at the University of Washington and the author of "The Master Algorithm", the worldwide bestseller introducing machine learning to a broad audience. He is a winner of the SIGKDD Innovation Award and the IJCAI John McCarthy Award, two of the highest honors in data science and AI, and a Fellow of the AAAS and AAAI. His research spans a wide variety of topics in machine learning, artificial intelligence, and data science. He helped start the fields of statistical relational AI, data stream mining, adversarial learning, machine learning for information integration, and influence maximization in social networks.
https://homes.cs.washington.edu/~pedrod/
https://www.amazon.com/Master-Algorithm-Ultimate-Learning-Machine-ebook/dp/B012271YB2
https://www.meetup.com/SF-Bay-ACM/events/275165417/
#DeepLearning #MachineLearning #GradientDescent
Видео Deep Networks Are Kernel Machines, Pedro Domingos канала San Francisco Bay ACM
Deep learning's successes are often attributed to its ability to automatically discover new representations of the data, rather than relying on handcrafted features like other learning methods. In this talk, however, Pedro Domingos will show that deep networks learned by the standard gradient descent algorithm are in fact mathematically approximately equivalent to kernel machines, a learning method that simply memorizes the data and uses it directly for prediction via a similarity function (the kernel). This greatly enhances the interpretability of deep network weights, by elucidating that they are effectively a superposition of the training examples. The network architecture incorporates knowledge of the target function into the kernel. The talk will include a discussion of both the main ideas behind this result and some of its more startling consequences for deep learning, kernel machines, and machine learning at large.
For his 2020 paper behind this talk, see: "Every Model Learned by Gradient Descent Is Approximately a Kernel Machine" at https://arxiv.org/abs/2012.00152
Pedro Domingos is a professor of computer science at the University of Washington and the author of "The Master Algorithm", the worldwide bestseller introducing machine learning to a broad audience. He is a winner of the SIGKDD Innovation Award and the IJCAI John McCarthy Award, two of the highest honors in data science and AI, and a Fellow of the AAAS and AAAI. His research spans a wide variety of topics in machine learning, artificial intelligence, and data science. He helped start the fields of statistical relational AI, data stream mining, adversarial learning, machine learning for information integration, and influence maximization in social networks.
https://homes.cs.washington.edu/~pedrod/
https://www.amazon.com/Master-Algorithm-Ultimate-Learning-Machine-ebook/dp/B012271YB2
https://www.meetup.com/SF-Bay-ACM/events/275165417/
#DeepLearning #MachineLearning #GradientDescent
Видео Deep Networks Are Kernel Machines, Pedro Domingos канала San Francisco Bay ACM
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
Deep Networks Are Kernel Machines (Paper Explained)The Master Algorithm | Pedro Domingos | Talks at Google[ECCV 2020] NeRF: Neural Radiance Fields (10 min talk)Neural Tangent Kernel: Convergence and Generalization in Neural NetworksWhat is... an elliptic curve?Convolutional Neural Networks | CNN | Kernel | Stride | Padding | Pooling | Flatten | FormulaSVM Kernels : Data Science ConceptsMIc'd Up | Part 2: Pedro Domingos in Conversation with Eric SchmidtThe Kernel Trick - THE MATH YOU SHOULD KNOW!The Next Hundred Years of Your Life | Pedro Domingos | TEDxLA2021's Biggest Breakthroughs in PhysicsJacobi Elliptic Function Differential EquationsAnalyzing Optimization and Generalization in Deep Learning via Trajectories of Gradient DescentQuanquan Gu: "Learning Over-parameterized Neural Networks: From Neural Tangent Kernel to Mean-fi..."Learning and Generalization in Over-parametrized Neural Networks, Going Beyond KernelsKernel TrickFlatten, Reshape, and Squeeze Explained - Tensors for Deep Learning with PyTorchWhat is cross entropy| cross entropy cost function|cross entropy lossWeightWatcher, an Open-Source Diagnostic Tool for Analyzing Deep Neural NetsKernels!