Stochastic Gradient Descent, Clearly Explained!!!

Even though Stochastic Gradient Descent sounds fancy, it is just a simple addition to "regular" Gradient Descent. This video sets up the problem that Stochastic Gradient Descent solves and then shows how it does it. Along the way, we discuss situations where Stochastic Gradient Descent is most useful, and some cool features that aren't that obvious.

NOTE: There is a small typo at 9:03. The values for the intercept and slope should be the most recent estimates, 0.86 and 0.68, instead of the original random values, 0 and 1.

NOTE: This StatQuest assumes you already understand "regular" Gradient Descent. If not, check out the 'Quest: https://youtu.be/sDv4f4s2SB8

When I was researching Stochastic Gradient Descent, I found a ton of cool websites that provided lots of details. Here are some of my favorites:

Sebastian Ruder has a nice write-up: http://ruder.io/optimizing-gradient-descent/

...as the Usupervised Feature Learning and Deep Learning Tutorial: http://deeplearning.stanford.edu/tutorial/supervised/OptimizationStochasticGradientDescent/

⭐ NOTE: When I code, I use Kite, a free AI-powered coding assistant that will help you code faster and smarter. The Kite plugin integrates with all the top editors and IDEs to give you smart completions and documentation while you’re typing. I love it! https://www.kite.com/get-kite/?utm_medium=referral&utm_source=youtube&utm_campaign=statquest&utm_content=description-only

For a complete index of all the StatQuest videos, check out:
https://statquest.org/video-index/

If you'd like to support StatQuest, please consider...
Patreon: https://www.patreon.com/statquest
...or...
YouTube Membership: https://www.youtube.com/channel/UCtYLUTtgS3k1Fg4y5tAhLbw/join

...a cool StatQuest t-shirt or sweatshirt (USA/Europe): https://teespring.com/stores/statquest
(everywhere):
https://www.redbubble.com/people/starmer/works/40421224-statquest-double-bam?asc=u&p=t-shirt

...buying one or two of my songs (or go large and get a whole album!)
https://joshuastarmer.bandcamp.com/

...or just donating to StatQuest!
https://www.paypal.me/statquest

Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
https://twitter.com/joshuastarmer

#statquest #sgd

Видео Stochastic Gradient Descent, Clearly Explained!!! канала StatQuest with Josh Starmer