Distilling the Knowledge in a Neural Network
This is the first and foundational paper that started the research area of Knowledge Distillation.
Knowledge Distillation is a study of methods and techniques to extract the information from a cumbersome model (also called the Teacher model) and provide it to a simpler model (also called the Student model). Student models are the ones that are used for inference (especially on resource-constrained devices) and are supposed to excel at both accuracy and speed of prediction
Link to the paper:
https://arxiv.org/abs/1503.02531
Link to the summary of the paper:
https://towardsdatascience.com/paper-summary-distilling-the-knowledge-in-a-neural-network-dc8efd9813cc
#KnowledgeDistillation
#deeplearning
#softmax
#machinelearning
Видео Distilling the Knowledge in a Neural Network канала Kapil Sachdeva
Knowledge Distillation is a study of methods and techniques to extract the information from a cumbersome model (also called the Teacher model) and provide it to a simpler model (also called the Student model). Student models are the ones that are used for inference (especially on resource-constrained devices) and are supposed to excel at both accuracy and speed of prediction
Link to the paper:
https://arxiv.org/abs/1503.02531
Link to the summary of the paper:
https://towardsdatascience.com/paper-summary-distilling-the-knowledge-in-a-neural-network-dc8efd9813cc
#KnowledgeDistillation
#deeplearning
#softmax
#machinelearning
Видео Distilling the Knowledge in a Neural Network канала Kapil Sachdeva
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
Matching patterns using Cross-Correlation | Essentials of MLEliminate Grid Sensitivity | Bag of Freebies (Yolov4) | Essentials of Object DetectionA Better Detection Head | Essentials of Object DetectionLet's make the Correlation Machine | Essentials of MLHow to Read & Make Graphical Models?FitNets: Hints for Thin Deep NetsVariational Autoencoder - VISUALLY EXPLAINED!Maximum Likelihood Estimation - THINK PROBABILITY FIRST!GIoU vs DIoU vs CIoU | Losses | Essentials of Object DetectionReshape,Permute,Squeeze,Unsqueeze made simple using einops | The GemsThe Battle of Polynomials | Towards Bayesian RegressionBounding Box Prediction | Yolo | Essentials of Object DetectionRejection Sampling - VISUALLY EXPLAINED with EXAMPLES!Monte Carlo Methods - VISUALLY EXPLAINED!Bounding Box Formats | Essentials of Object DetectionSum Rule, Product Rule, Joint & Marginal Probability - CLEARLY EXPLAINED with EXAMPLES!Normalizing Flows - Motivations, The Big Idea, & Essential FoundationsFeature Pyramid Network | Neck | Essentials of Object DetectionReparameterization Trick - WHY & BUILDING BLOCKS EXPLAINED!KL Divergence - CLEARLY EXPLAINED!Softmax (with Temperature) | Essentials of ML