Mixup in PyTorch

In this video, we implement the (input) mixup and manifold mixup. They are regularization techniques proposed in the papers "mixup: Beyond Empirical Risk Minimization" and "Manifold Mixup: Better Representations by Interpolating Hidden States". We investigate how these two schemes compare against more mainstream regularization methods like dropout and weight decay.

Paper (Input mixup): https://arxiv.org/abs/1710.09412
Paper (Manifold mixup): https://arxiv.org/abs/1806.05236
Code (Input mixup): https://github.com/facebookresearch/mixup-cifar10
Code (Manifold mixup): https://github.com/vikasverma1077/manifold_mixup

Code from this video: https://github.com/jankrepl/mildlyoverfitted/tree/master/github_adventures/mixup

00:00 Intro
00:53 Disclaimer
01:14 Input mixup overview
01:53 Manifold mixup overview
02:48 Outline of the code
03:14 Multilayer perceptron (modified for mixup)
06:29 X, y dataset wrapper
07:29 Spiral dataset generator
09:56 Plotting function
12:10 CLI arguments
13:26 Training preparations
15:18 Mixup + training logic
17:49 Evaluation + tracking logic
18:48 Preparing experiments
19:29 Results: Metrics
21:14 Results: Plots
23:44 Outro
If you have any video suggestions or you just wanna chat feel free to join the discord server: https://discord.gg/a8Va9tZsG5

Twitter: https://twitter.com/moverfitted

Credits logo animation
Title: Conjungation · Author: Uncle Milk · Source: https://soundcloud.com/unclemilk · License: https://creativecommons.org/licenses/... · Download (9MB): https://auboutdufil.com/?id=600

Видео Mixup in PyTorch канала mildlyoverfitted