Zephyr 7B beta - How much does DPO really help?

Colab Full model with DPO: https://drp.li/QYAnu
Colab with SFT Only: https://drp.li/HAvSc

For more tutorials on using LLMs and building Agents, check out my Patreon:
Patreon: https://www.patreon.com/SamWitteveen
Twitter: https://twitter.com/Sam_Witteveen

My Links:
Linkedin: https://www.linkedin.com/in/samwitteveen/

Github:
https://github.com/samwit/langchain-tutorials (updated)
https://github.com/samwit/llm-tutorials

Timestamps
00:00 Intro
00:15 Zephyr 7B - Model on HF
01:04 Zephyr 7B -Beta Technical Paper
01:49 MT Bench
02:07 AlpacaEval
02:28 UltraChat Dataset
02:48 Zephyr 7B-Beta Flaws
03:21 UltraFeedback Dataset
05:26 Code Time
05:35 Full Model with DPO
08:44 Model with SFT Only
12:11 Alignment Notebook

Видео Zephyr 7B beta - How much does DPO really help? канала Sam Witteveen