AdaBoost, Clearly Explained

AdaBoost is one of those machine learning methods that seems so much more confusing than it really is. It's really just a simple twist on decision trees and random forests.

⭐ NOTE: When I code, I use Kite, a free AI-powered coding assistant that will help you code faster and smarter. The Kite plugin integrates with all the top editors and IDEs to give you smart completions and documentation while you’re typing. I love it! https://www.kite.com/get-kite/?utm_medium=referral&utm_source=youtube&utm_campaign=statquest&utm_content=description-only

NOTE: This video assumes you already know about Decision Trees...
https://youtu.be/_L39rN6gz7Y
...and Random Forests....
https://youtu.be/J4Wdy0Wc_xQ

For a complete index of all the StatQuest videos, check out:
https://statquest.org/video-index/

Sources:
The original AdaBoost paper by Robert E. Schapire and Yoav Freund
https://www.sciencedirect.com/science/article/pii/S002200009791504X

And a follow up by co-created Schapire:
http://rob.schapire.net/papers/explaining-adaboost.pdf

The idea of using the weights to resample the original dataset comes from Boosting Foundations and Algorithms, by Robert E. Schapire and Yoav Freund
https://mitpress.mit.edu/books/boosting

Lastly, Chris McCormick's tutorial was super helpful:
http://mccormickml.com/2013/12/13/adaboost-tutorial/

If you'd like to support StatQuest, please consider...
Patreon: https://www.patreon.com/statquest
...or...
YouTube Membership: https://www.youtube.com/channel/UCtYLUTtgS3k1Fg4y5tAhLbw/join

...a cool StatQuest t-shirt or sweatshirt (USA/Europe): https://teespring.com/stores/statquest
(everywhere):
https://www.redbubble.com/people/starmer/works/40421224-statquest-double-bam?asc=u&p=t-shirt

...buying one or two of my songs (or go large and get a whole album!)
https://joshuastarmer.bandcamp.com/

...or just donating to StatQuest!
https://www.paypal.me/statquest

Lastly, if you want to keep up with me as I research and create new StatQuests, follow me on twitter:
https://twitter.com/joshuastarmer

0:00 Awesome song and introduction
0:56 The three main ideas behind AdaBoost
3:30 Review of the three main ideas
3:58 Building a stump with the GINI index
6:27 Determining the Amount of Say for a stump
10:45 Updating sample weights
14:47 Normalizing the sample weights
15:32 Using the normalized weights to make the second stump
19:06 Using stumps to make classifications
19:51 Review of the three main ideas behind AdaBoost
#statquest #adaboost

Видео AdaBoost, Clearly Explained канала StatQuest with Josh Starmer