The Lottery Ticket Hypothesis with Jonathan Frankle
In this episode of Machine Learning Street Talk, we chat with Jonathan Frankle, author of The Lottery Ticket Hypothesis. Frankle has continued researching Sparse Neural Networks, Pruning, and Lottery Tickets leading to some really exciting follow-on papers! This chat discusses some of these papers such as Linear Mode Connectivity, Comparing and Rewinding and Fine-tuning in Neural Network Pruning, and more (full list of papers linked below). We also chat about how Jonathan got into Deep Learning research, his Information Diet, and work on developing Technology Policy for Artificial Intelligence!
This was a really fun chat, I hope you enjoy listening to it and learn something from it!
Thanks for watching and please subscribe!
Huge thanks to everyone on r/MachineLearning who asked questions!
Paper Links discussed in the chat:
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks: https://arxiv.org/abs/1803.03635
Linear Mode Connectivity and the Lottery Ticket Hypothesis: https://arxiv.org/abs/1912.05671
Dissecting Pruned Neural Networks: https://arxiv.org/abs/1907.00262
Training BatchNorm and Only BatchNorm: On the Expressive Power of Random Features in CNNs: https://arxiv.org/abs/2003.00152
What is the State of Neural Network Pruning? https://arxiv.org/abs/2003.03033
The Early Phase of Neural Network Training: https://arxiv.org/abs/2002.10365
Comparing Rewinding and Fine-tuning in Neural Network Pruning: https://arxiv.org/abs/2003.02389
(Also Mentioned)
Block-Sparse GPU Kernels: https://openai.com/blog/block-sparse-gpu-kernels/
Balanced Sparsity for Efficient DNN Inference on GPU: https://arxiv.org/pdf/1811.00206.pdf
Playing the Lottery with Rewards and Multiple Languages: Lottery Tickets in RL and NLP: https://arxiv.org/pdf/1906.02768.pdf
r/MachineLearning question list: https://www.reddit.com/r/MachineLearning/comments/g9jqe0/d_lottery_ticket_hypothesis_ask_the_author_a/ (edited)
#machinelearning #deeplearning
Видео The Lottery Ticket Hypothesis with Jonathan Frankle канала Machine Learning Street Talk
This was a really fun chat, I hope you enjoy listening to it and learn something from it!
Thanks for watching and please subscribe!
Huge thanks to everyone on r/MachineLearning who asked questions!
Paper Links discussed in the chat:
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks: https://arxiv.org/abs/1803.03635
Linear Mode Connectivity and the Lottery Ticket Hypothesis: https://arxiv.org/abs/1912.05671
Dissecting Pruned Neural Networks: https://arxiv.org/abs/1907.00262
Training BatchNorm and Only BatchNorm: On the Expressive Power of Random Features in CNNs: https://arxiv.org/abs/2003.00152
What is the State of Neural Network Pruning? https://arxiv.org/abs/2003.03033
The Early Phase of Neural Network Training: https://arxiv.org/abs/2002.10365
Comparing Rewinding and Fine-tuning in Neural Network Pruning: https://arxiv.org/abs/2003.02389
(Also Mentioned)
Block-Sparse GPU Kernels: https://openai.com/blog/block-sparse-gpu-kernels/
Balanced Sparsity for Efficient DNN Inference on GPU: https://arxiv.org/pdf/1811.00206.pdf
Playing the Lottery with Rewards and Multiple Languages: Lottery Tickets in RL and NLP: https://arxiv.org/pdf/1906.02768.pdf
r/MachineLearning question list: https://www.reddit.com/r/MachineLearning/comments/g9jqe0/d_lottery_ticket_hypothesis_ask_the_author_a/ (edited)
#machinelearning #deeplearning
Видео The Lottery Ticket Hypothesis with Jonathan Frankle канала Machine Learning Street Talk
Показать
Комментарии отсутствуют
Информация о видео
2 мая 2020 г. 23:17:15
01:26:44
Другие видео канала
The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural NetworksJ. Frankle & M. Carbin: The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks#78 - Prof. NOAM CHOMSKY (Special Edition)Kernels!Exploring the Limits of Transfer Learning with a Unified Text-to-Text TransformerNEARLY PERFECT! 2022 Audi RS5 Review#71 - ZAK JOST (Graph Neural Networks + Geometric DL) [UNPLUGGED]How Mega Millions And Powerball Jackpots Grew So LargeJonathan Frankle — The Lottery Ticket Hypothesis: On Sparse, Trainable Neural Networks#61: Prof. YANN LECUN: Interpolation, Extrapolation and Linearisation (w/ Dr. Randall Balestriero)The Lottery Ticket Hypothesis Explained!Kaggle, ML Community / Engineering14-Times Lottery Winner Finally Reveals His Secret#72 Prof. KEN STANLEY 2.0 - On Art and Subjectivity [UNPLUGGED]Lottery Prediction Trick REVEALED!!!Predicting Football Matches Using Data With Jordan Tigani - Strata Europe 2014Machine Learning with No Code#75 - Emergence [Special Edition] with Dr. DANIELE GRATTAROLARiemann Hypothesis - Numberphile