Vanishing & Exploding Gradient explained | A problem resulting from backpropagation
Let's discuss a problem that creeps up time-and-time during the training process of an artificial neural network. This is the problem of unstable gradients, and is most popularly referred to as the vanishing gradient problem.
We're first going to answer the question, what is the vanishing gradient problem, anyway? Here, we'll cover the idea conceptually. We'll then move our discussion to talking about how this problem occurs. Then, with the understanding that we'll have developed up to this point, we'll discuss the problem of exploding gradients, which we'll see is actually very similar to the vanishing gradient problem, and so we'll be able to take what we learned about that problem and apply it to this new one.
🕒🦎 VIDEO SECTIONS 🦎🕒
00:00 Welcome to DEEPLIZARD - Go to deeplizard.com for learning resources
00:28 Gradient review
01:18 Agenda
01:45 The vanishing gradient problem
03:27 The cause of the vanishing gradients
05:30 Exploding gradient
07:13 Collective Intelligence and the DEEPLIZARD HIVEMIND
💥🦎 DEEPLIZARD COMMUNITY RESOURCES 🦎💥
👋 Hey, we're Chris and Mandy, the creators of deeplizard!
👀 CHECK OUT OUR VLOG:
🔗 https://youtube.com/deeplizardvlog
👉 Check out the blog post and other resources for this video:
🔗 https://deeplizard.com/learn/video/qO_NLVjD6zE
💻 DOWNLOAD ACCESS TO CODE FILES
🤖 Available for members of the deeplizard hivemind:
🔗 https://deeplizard.com/resources
🧠 Support collective intelligence, join the deeplizard hivemind:
🔗 https://deeplizard.com/hivemind
🤜 Support collective intelligence, create a quiz question for this video:
🔗 https://deeplizard.com/create-quiz-question
🚀 Boost collective intelligence by sharing this video on social media!
❤️🦎 Special thanks to the following polymaths of the deeplizard hivemind:
Tammy
Prash
Zach Wimpee
👀 Follow deeplizard:
Our vlog: https://youtube.com/deeplizardvlog
Facebook: https://facebook.com/deeplizard
Instagram: https://instagram.com/deeplizard
Twitter: https://twitter.com/deeplizard
Patreon: https://patreon.com/deeplizard
YouTube: https://youtube.com/deeplizard
🎓 Deep Learning with deeplizard:
Fundamental Concepts - https://deeplizard.com/learn/video/gZmobeGL0Yg
Beginner Code - https://deeplizard.com/learn/video/RznKVRTFkBY
Intermediate Code - https://deeplizard.com/learn/video/v5cngxo4mIg
Advanced Deep RL - https://deeplizard.com/learn/video/nyjbcRQ-uQ8
🎓 Other Courses:
Data Science - https://deeplizard.com/learn/video/d11chG7Z-xk
Trading - https://deeplizard.com/learn/video/ZpfCK_uHL9Y
🛒 Check out products deeplizard recommends on Amazon:
🔗 https://amazon.com/shop/deeplizard
📕 Get a FREE 30-day Audible trial and 2 FREE audio books using deeplizard's link:
🔗 https://amzn.to/2yoqWRn
🎵 deeplizard uses music by Kevin MacLeod
🔗 https://youtube.com/channel/UCSZXFhRIx6b0dFX3xS8L1yQ
🔗 http://incompetech.com/
❤️ Please use the knowledge gained from deeplizard content for good, not evil.
Видео Vanishing & Exploding Gradient explained | A problem resulting from backpropagation канала deeplizard
We're first going to answer the question, what is the vanishing gradient problem, anyway? Here, we'll cover the idea conceptually. We'll then move our discussion to talking about how this problem occurs. Then, with the understanding that we'll have developed up to this point, we'll discuss the problem of exploding gradients, which we'll see is actually very similar to the vanishing gradient problem, and so we'll be able to take what we learned about that problem and apply it to this new one.
🕒🦎 VIDEO SECTIONS 🦎🕒
00:00 Welcome to DEEPLIZARD - Go to deeplizard.com for learning resources
00:28 Gradient review
01:18 Agenda
01:45 The vanishing gradient problem
03:27 The cause of the vanishing gradients
05:30 Exploding gradient
07:13 Collective Intelligence and the DEEPLIZARD HIVEMIND
💥🦎 DEEPLIZARD COMMUNITY RESOURCES 🦎💥
👋 Hey, we're Chris and Mandy, the creators of deeplizard!
👀 CHECK OUT OUR VLOG:
🔗 https://youtube.com/deeplizardvlog
👉 Check out the blog post and other resources for this video:
🔗 https://deeplizard.com/learn/video/qO_NLVjD6zE
💻 DOWNLOAD ACCESS TO CODE FILES
🤖 Available for members of the deeplizard hivemind:
🔗 https://deeplizard.com/resources
🧠 Support collective intelligence, join the deeplizard hivemind:
🔗 https://deeplizard.com/hivemind
🤜 Support collective intelligence, create a quiz question for this video:
🔗 https://deeplizard.com/create-quiz-question
🚀 Boost collective intelligence by sharing this video on social media!
❤️🦎 Special thanks to the following polymaths of the deeplizard hivemind:
Tammy
Prash
Zach Wimpee
👀 Follow deeplizard:
Our vlog: https://youtube.com/deeplizardvlog
Facebook: https://facebook.com/deeplizard
Instagram: https://instagram.com/deeplizard
Twitter: https://twitter.com/deeplizard
Patreon: https://patreon.com/deeplizard
YouTube: https://youtube.com/deeplizard
🎓 Deep Learning with deeplizard:
Fundamental Concepts - https://deeplizard.com/learn/video/gZmobeGL0Yg
Beginner Code - https://deeplizard.com/learn/video/RznKVRTFkBY
Intermediate Code - https://deeplizard.com/learn/video/v5cngxo4mIg
Advanced Deep RL - https://deeplizard.com/learn/video/nyjbcRQ-uQ8
🎓 Other Courses:
Data Science - https://deeplizard.com/learn/video/d11chG7Z-xk
Trading - https://deeplizard.com/learn/video/ZpfCK_uHL9Y
🛒 Check out products deeplizard recommends on Amazon:
🔗 https://amazon.com/shop/deeplizard
📕 Get a FREE 30-day Audible trial and 2 FREE audio books using deeplizard's link:
🔗 https://amzn.to/2yoqWRn
🎵 deeplizard uses music by Kevin MacLeod
🔗 https://youtube.com/channel/UCSZXFhRIx6b0dFX3xS8L1yQ
🔗 http://incompetech.com/
❤️ Please use the knowledge gained from deeplizard content for good, not evil.
Видео Vanishing & Exploding Gradient explained | A problem resulting from backpropagation канала deeplizard
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
![Weight Initialization explained | A way to reduce the vanishing gradient problem](https://i.ytimg.com/vi/8krd5qKVw-Q/default.jpg)
![Learnable Parameters in a Convolutional Neural Network (CNN) explained](https://i.ytimg.com/vi/gmBfb6LNnZs/default.jpg)
![Tutorial 7- Vanishing Gradient Problem](https://i.ytimg.com/vi/JIWXbzRXk1I/default.jpg)
![Vanishing and exploding gradients | Deep Learning Tutorial 35 (Tensorflow, Keras & Python)](https://i.ytimg.com/vi/qowp6SQ9_Oo/default.jpg)
![An Old Problem - Ep. 5 (Deep Learning SIMPLIFIED)](https://i.ytimg.com/vi/SKMpmAOUa2Q/default.jpg)
![Vanishing/Exploding Gradients (C2W1L10)](https://i.ytimg.com/vi/qhXZsFVxGKo/default.jpg)
![](https://i.ytimg.com/vi/mMXqI2BDVO8/default.jpg)
![RNN W1L08 : Vanishing gradients with RNNs](https://i.ytimg.com/vi/3Hn_hEPtciQ/default.jpg)
![Vanishing Gradients and Activation Functions - Intro to Deep Learning using TensorFlow #8](https://i.ytimg.com/vi/kGAo32JgY48/default.jpg)
![Exploding Gradient and Vanishing Gradient problem in deep neural network|Deep learning tutorial](https://i.ytimg.com/vi/IBODsB4q8cQ/default.jpg)
![Regularization in a Neural Network explained](https://i.ytimg.com/vi/iuJgyiS7BKM/default.jpg)
![Stochastic Gradient Descent, Clearly Explained!!!](https://i.ytimg.com/vi/vMh0zPT0tLI/default.jpg)
![Tutorial 32- Problems In Simple Recurrent Neural Network](https://i.ytimg.com/vi/mDaEfPgwtgo/default.jpg)
![CS 152 NN—17: Gradient Clipping](https://i.ytimg.com/vi/_-CZr06R5CQ/default.jpg)
![Fine-tuning a Neural Network explained](https://i.ytimg.com/vi/5T-iXNNiwIs/default.jpg)
![What is backpropagation really doing? | Chapter 3, Deep learning](https://i.ytimg.com/vi/Ilg3gGewQ5U/default.jpg)
![Tutorial 8- Exploding Gradient Problem in Neural Network](https://i.ytimg.com/vi/IJ9atfxFjOQ/default.jpg)
![Batch Normalization (“batch norm”) explained](https://i.ytimg.com/vi/dXB-KQYkzNU/default.jpg)
![Deep Learning(CS7015): Lec 14.3 How LSTMs avoid the problem of vanishing gradients](https://i.ytimg.com/vi/PMv8C-Ws1b8/default.jpg)
![Backpropagation explained | Part 1 - The intuition](https://i.ytimg.com/vi/XE3krf3CQls/default.jpg)