What is LoRA? Low-Rank Adaptation for finetuning LLMs EXPLAINED
How does LoRA work? Low-Rank Adaptation for Parameter-Efficient LLM Finetuning explained. Works for any other neural network as well, not just for LLMs.
📜 „Lora: Low-rank adaptation of large language models“ Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L. and Chen, W., 2021. https://arxiv.org/abs/2106.09685
📚 https://sebastianraschka.com/blog/2023/llm-finetuning-lora.html
📽️ LoRA implementation: https://youtu.be/iYr1xZn26R8
Thanks to our Patrons who support us in Tier 2, 3, 4: 🙏
Dres. Trost GbR, Siltax, Vignesh Valliappan, Mutual Information, Kshitij
Outline:
00:00 LoRA explained
00:59 Why finetuning LLMs is costly
01:44 How LoRA works
03:45 Low-rank adaptation
06:14 LoRA vs other approaches
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔥 Optionally, pay us a coffee to help with our Coffee Bean production! ☕
Patreon: https://www.patreon.com/AICoffeeBreak
Ko-fi: https://ko-fi.com/aicoffeebreak
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔗 Links:
AICoffeeBreakQuiz: https://www.youtube.com/c/AICoffeeBreak/community
Twitter: https://twitter.com/AICoffeeBreak
Reddit: https://www.reddit.com/r/AICoffeeBreak/
YouTube: https://www.youtube.com/AICoffeeBreak
#AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #research
Music 🎵 : Meadows - Ramzoid
Video editing: Nils Trost
Видео What is LoRA? Low-Rank Adaptation for finetuning LLMs EXPLAINED канала AI Coffee Break with Letitia
📜 „Lora: Low-rank adaptation of large language models“ Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L. and Chen, W., 2021. https://arxiv.org/abs/2106.09685
📚 https://sebastianraschka.com/blog/2023/llm-finetuning-lora.html
📽️ LoRA implementation: https://youtu.be/iYr1xZn26R8
Thanks to our Patrons who support us in Tier 2, 3, 4: 🙏
Dres. Trost GbR, Siltax, Vignesh Valliappan, Mutual Information, Kshitij
Outline:
00:00 LoRA explained
00:59 Why finetuning LLMs is costly
01:44 How LoRA works
03:45 Low-rank adaptation
06:14 LoRA vs other approaches
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔥 Optionally, pay us a coffee to help with our Coffee Bean production! ☕
Patreon: https://www.patreon.com/AICoffeeBreak
Ko-fi: https://ko-fi.com/aicoffeebreak
▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀
🔗 Links:
AICoffeeBreakQuiz: https://www.youtube.com/c/AICoffeeBreak/community
Twitter: https://twitter.com/AICoffeeBreak
Reddit: https://www.reddit.com/r/AICoffeeBreak/
YouTube: https://www.youtube.com/AICoffeeBreak
#AICoffeeBreak #MsCoffeeBean #MachineLearning #AI #research
Music 🎵 : Meadows - Ramzoid
Video editing: Nils Trost
Видео What is LoRA? Low-Rank Adaptation for finetuning LLMs EXPLAINED канала AI Coffee Break with Letitia
Показать
Комментарии отсутствуют
Информация о видео
18 сентября 2023 г. 18:00:12
00:08:22
Другие видео канала
Are ChatBots their own death? | Training on Generated Data Makes Models Forget – Paper explainedThe first law on AI regulation | The EU AI ActSay that 3 times in a row. 😅Author Interviews, Poster Highlights, Summary of the ACL 2023 Toronto NLPChatGPT ist not an intelligent agent. It is a cultural technology. – Gopnik KeynoteDo LLMs understand? Jay Alammar's TLDR of Geoffrey Hinton ACL2023 Keynote[Own work] MM-SHAP to measure modality contributionsEight Things to Know about Large Language ModelsSpeaking about AI is hard, even for humans | AI Coffee Break BloopersMoral Self-Correction in Large Language Models | paper explainedAI beats us at another game: STRATEGO | DeepNash paper explainedWhy ChatGPT fails | Language Model Limitations EXPLAINED"Watermarking Language Models" paper and GPTZero EXPLAINED | How to detect text by ChatGPT?Training learned optimizers: VeLO paper EXPLAINEDChatGPT vs Sparrow - Battle of ChatbotsPaella: Text to image FASTER than diffusion models | Paella paper explainedGenerate long form video with Transformers | Phenaki from Google Brain explainedMovie Diffusion explained | Make-a-Video from MetaAI and Imagen Video from Google BrainBeyond neural scaling laws – Paper ExplainedHow does Stable Diffusion work? – Latent Diffusion Models EXPLAINED