Learning Representations: A Challenge for Learning Theory, COLT 2013 | Yann LeCun, NYU
Slides: http://videolectures.net/site/normal_dl/tag=800934/colt2013_lecun_theory_01.pdf
Perceptual tasks such as vision and audition require the construction of good features, or good internal representations of the input. Deep Learning designates a set of supervised and unsupervised methods to construct feature hierarchies automatically by training systems composed of multiple stages of trainable modules.The recent history of OCR, speech recognition, and image analysis indicates that deep learning systems yield higher accuracy than systems that rely on hand-crafted features or "shallow" architectures whenever more training data and more computational resources become available. Deep learning systems, particularly convolutional nets, hold the performances record in a wide variety of benchmarks and competition, including object recognition in image, semantic image labeling (2D and 3D), acoustic modeling for speech recognition, drug design, handwriting recognition, pedestrian detection, road sign recognition, etc. The most recent speech recognition and image analysis systems deployed by Google, IBM, Microsoft, Baidu, NEC and others all use deep learning and many use convolutional nets.While the practical successes of deep learning are numerous, so are the theoretical questions that surround it. What can circuit complexity theory tell us about deep architectures with their multiple sequential steps of computation, compared to, say, kernel machines with simple kernels that have only two steps? What can learning theory tell us about unsupervised feature learning? What can theory tell us about the properties of deep architectures composed of layers that expand the dimension of their input (e.g. like sparse coding), followed by layers that reduce it (e.g. like pooling)? What can theory tell us about the properties of the non-convex objective functions that arise in deep learning? Why is it that the best-performing deep learning systems happen to be ridiculously over-parameterized with regularization so aggressive that it borders on genocide?
Видео Learning Representations: A Challenge for Learning Theory, COLT 2013 | Yann LeCun, NYU канала Preserve Knowledge
Perceptual tasks such as vision and audition require the construction of good features, or good internal representations of the input. Deep Learning designates a set of supervised and unsupervised methods to construct feature hierarchies automatically by training systems composed of multiple stages of trainable modules.The recent history of OCR, speech recognition, and image analysis indicates that deep learning systems yield higher accuracy than systems that rely on hand-crafted features or "shallow" architectures whenever more training data and more computational resources become available. Deep learning systems, particularly convolutional nets, hold the performances record in a wide variety of benchmarks and competition, including object recognition in image, semantic image labeling (2D and 3D), acoustic modeling for speech recognition, drug design, handwriting recognition, pedestrian detection, road sign recognition, etc. The most recent speech recognition and image analysis systems deployed by Google, IBM, Microsoft, Baidu, NEC and others all use deep learning and many use convolutional nets.While the practical successes of deep learning are numerous, so are the theoretical questions that surround it. What can circuit complexity theory tell us about deep architectures with their multiple sequential steps of computation, compared to, say, kernel machines with simple kernels that have only two steps? What can learning theory tell us about unsupervised feature learning? What can theory tell us about the properties of deep architectures composed of layers that expand the dimension of their input (e.g. like sparse coding), followed by layers that reduce it (e.g. like pooling)? What can theory tell us about the properties of the non-convex objective functions that arise in deep learning? Why is it that the best-performing deep learning systems happen to be ridiculously over-parameterized with regularization so aggressive that it borders on genocide?
Видео Learning Representations: A Challenge for Learning Theory, COLT 2013 | Yann LeCun, NYU канала Preserve Knowledge
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
Meet Geoffrey Hinton, U of T's Godfather of Deep LearningDavid Duvenaud | Reflecting on Neural ODEs | NeurIPS 2019Yoshua Bengio | From System 1 Deep Learning to System 2 Deep Learning | NeurIPS 2019NIPS 2017 Test of Time Award "Machine learning has become alchemy.” | Ali Rahimi, GoogleSemantic Segmentation using Adversarial Networks, NIPS 2016 | Pauline Luc, Facebook AI ResearchNeurIPS 2019 Test of Time Award - Lin XiaoHeroes of Deep Learning: Andrew Ng interviews Yann LeCunYann LeCun: Turing Award Lecture "The Deep Learning Revolution: The Sequel"How AI Powers Self-Driving Tesla with Elon Musk and Andrej KarpathyHipótesis del Monitor (Krashen)Geoffrey Hinton: Turing Award Lecture "The Deep Learning Revolution"Tesla AI Andrej Karpathy on Scalability in Autonomous DrivingMicrosoft CEO Satya Nadella CVPR 2020Kuver Sinha (Syracuse) HEPEC Seminar 10.18.13Kristan Jensen (Stony Brook) HET Seminar 10.21.132022.6.23 木5 早稻田 Servant Leadership by Professor WeakleyLas cinco teorías del Dr. Krashen.Realignment or dealignment? Survey perspectives on the 2022 Federal Election