Weight Initialization explained | A way to reduce the vanishing gradient problem

Let's talk about how the weights in an artificial neural network are initialized, how this initialization affects the training process, and what YOU can do about it!

To kick off our discussion on weight initialization, we're first going to discuss how these weights are initialized, and how these initialized values might negatively affect the training process. We'll see that these randomly initialized weights actually contribute to the vanishing and exploding gradient problem we covered in the last video.

With this in mind, we'll then explore what we can do to influence how this initialization occurs. We'll see how Xavier initialization (also called Glorot initialization) can help combat this problem. Then, we'll see how we can specify how the weights for a given model are initialized in code using the kernel_initializer parameter for a given layer in Keras.

Reference to original paper by Xavier Glorot and Yoshua Bengio:
http://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf

🕒🦎 VIDEO SECTIONS 🦎🕒

00:00 Welcome to DEEPLIZARD - Go to deeplizard.com for learning resources
00:30 Help deeplizard add video timestamps - See example in the description
09:42 Collective Intelligence and the DEEPLIZARD HIVEMIND

💥🦎 DEEPLIZARD COMMUNITY RESOURCES 🦎💥

👋 Hey, we're Chris and Mandy, the creators of deeplizard!
👀 CHECK OUT OUR VLOG:
🔗 https://youtube.com/deeplizardvlog

👉 Check out the blog post and other resources for this video:
🔗 https://deeplizard.com/learn/video/8krd5qKVw-Q

💻 DOWNLOAD ACCESS TO CODE FILES
🤖 Available for members of the deeplizard hivemind:
🔗 https://deeplizard.com/resources

🧠 Support collective intelligence, join the deeplizard hivemind:
🔗 https://deeplizard.com/hivemind

🤜 Support collective intelligence, create a quiz question for this video:
🔗 https://deeplizard.com/create-quiz-question

🚀 Boost collective intelligence by sharing this video on social media!

❤️🦎 Special thanks to the following polymaths of the deeplizard hivemind:
Tammy
Prash
Zach Wimpee

👀 Follow deeplizard:
Our vlog: https://youtube.com/deeplizardvlog
Facebook: https://facebook.com/deeplizard
Instagram: https://instagram.com/deeplizard
Twitter: https://twitter.com/deeplizard
Patreon: https://patreon.com/deeplizard
YouTube: https://youtube.com/deeplizard

🎓 Deep Learning with deeplizard:
Fundamental Concepts - https://deeplizard.com/learn/video/gZmobeGL0Yg
Beginner Code - https://deeplizard.com/learn/video/RznKVRTFkBY
Intermediate Code - https://deeplizard.com/learn/video/v5cngxo4mIg
Advanced Deep RL - https://deeplizard.com/learn/video/nyjbcRQ-uQ8

🎓 Other Courses:
Data Science - https://deeplizard.com/learn/video/d11chG7Z-xk
Trading - https://deeplizard.com/learn/video/ZpfCK_uHL9Y

🛒 Check out products deeplizard recommends on Amazon:
🔗 https://amazon.com/shop/deeplizard

📕 Get a FREE 30-day Audible trial and 2 FREE audio books using deeplizard's link:
🔗 https://amzn.to/2yoqWRn

🎵 deeplizard uses music by Kevin MacLeod
🔗 https://youtube.com/channel/UCSZXFhRIx6b0dFX3xS8L1yQ
🔗 http://incompetech.com/

❤️ Please use the knowledge gained from deeplizard content for good, not evil.

Видео Weight Initialization explained | A way to reduce the vanishing gradient problem канала deeplizard