Inferencing and training LLMs with less GPUs - Hung Tran
From the January 2024 Machine Learning & AI Meetup: www.meetup.com/machine-learning-ai-meetup/
Talk Description: This talk dives into clever ways to run Large Language Models (LLMs) with fewer GPUs, tackling both inference and training stages. For inference on limited hardware, we'll explore techniques like model partitioning/offloading and quantization to shrink their memory footprint. To slash training costs, we'll delve into ZeRO memory optimization, LoRA, and prompt-tuning, paving the way for more efficient model development. And to cap it off, we'll showcase running LLMs solely on a personal laptop with CPUs and RAM, demonstrating the potential to democratise access to these powerful language models.
Speaker Bio: As an advancing Ph.D. student at Deakin University, specializing in video understanding, Hung Tran is a pioneer in applying deep learning and large language models to complex video data. His research involves processing video data, designing deep learning architectures and training them on multiple GPUs to uncover hidden patterns and predict future events. Recently, he explored how the inductive biases of large language models can enhance video analysis. Beyond research, he is a passionate coder with some experiences in backend web development.
Link to Slides: https://docs.google.com/presentation/d/19EwGnMseSaP4LzTEPMCYZyQtjGIB9P1HPT_FtgK9rUQ/edit?usp=share_link
Видео Inferencing and training LLMs with less GPUs - Hung Tran канала Machine Learning and AI Meetup
Talk Description: This talk dives into clever ways to run Large Language Models (LLMs) with fewer GPUs, tackling both inference and training stages. For inference on limited hardware, we'll explore techniques like model partitioning/offloading and quantization to shrink their memory footprint. To slash training costs, we'll delve into ZeRO memory optimization, LoRA, and prompt-tuning, paving the way for more efficient model development. And to cap it off, we'll showcase running LLMs solely on a personal laptop with CPUs and RAM, demonstrating the potential to democratise access to these powerful language models.
Speaker Bio: As an advancing Ph.D. student at Deakin University, specializing in video understanding, Hung Tran is a pioneer in applying deep learning and large language models to complex video data. His research involves processing video data, designing deep learning architectures and training them on multiple GPUs to uncover hidden patterns and predict future events. Recently, he explored how the inductive biases of large language models can enhance video analysis. Beyond research, he is a passionate coder with some experiences in backend web development.
Link to Slides: https://docs.google.com/presentation/d/19EwGnMseSaP4LzTEPMCYZyQtjGIB9P1HPT_FtgK9rUQ/edit?usp=share_link
Видео Inferencing and training LLMs with less GPUs - Hung Tran канала Machine Learning and AI Meetup
Показать
Комментарии отсутствуют
Информация о видео
20 февраля 2024 г. 4:54:10
00:46:14
Другие видео канала
![FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence-Covered by Adel Foda](https://i.ytimg.com/vi/gSMI5wZHe9w/default.jpg)
![Mat Kelcey — Deep RL for Robotics (FIXED)](https://i.ytimg.com/vi/EydQxFR4pzU/default.jpg)
![A Framework for Understanding Unintended Consequences of Machine Learning — Laura Summers](https://i.ytimg.com/vi/IRWaE1u9mDM/default.jpg)
![Mostafa Rizk - Teaching Autonomous Agents to Work Together](https://i.ytimg.com/vi/ToVLiHXO7XU/default.jpg)
![Probability as Logical Inference: a dramatic reading of E.T. Jaynes' textbook - Benji Metha](https://i.ytimg.com/vi/zOeOgXbC5hE/default.jpg)
![Audrey Lewis - Generating Process Discovery Workflows from Model-Free Reinforcement Learning Agents](https://i.ytimg.com/vi/seFKiLatpSE/default.jpg)
![Using Cross Entropy for Metric Learning — Mat Kelcey — May Meetup](https://i.ytimg.com/vi/Jb4Ewl5RzkI/default.jpg)
![AI: Turning Points and the Road Ahead: The Journey Thus Far - Dr. Lito Cruz](https://i.ytimg.com/vi/5fkJMFwr21E/default.jpg)
![You Should Know About: Compressive Sensing - Alex Tritt](https://i.ytimg.com/vi/lHD3-qVde24/default.jpg)
![The Map Interpretation of Attention — Mat Kelcey](https://i.ytimg.com/vi/26SzCzf-SaI/default.jpg)
![Bayesian Programming with JAX + NumPyro — Andy Kitchen](https://i.ytimg.com/vi/CecuWGpoztw/default.jpg)
![Mat Kelcey — Deep RL for Robotics](https://i.ytimg.com/vi/d2FP1CMEI94/default.jpg)
![Causal Induction from Visual Observations for Goal Directed Tasks — Lizzie Silver](https://i.ytimg.com/vi/cDXaAq0noPs/default.jpg)
![Practical Advice for Working with Large Language Models - Ned Letcher](https://i.ytimg.com/vi/FYyDw_s-Zmo/default.jpg)
![Causal Wizard: Helping subject matter experts get causal insights from historical data](https://i.ytimg.com/vi/Q2TEdpYO-mw/default.jpg)
![Elizabeth Silver — Causality and Causal Discovery](https://i.ytimg.com/vi/51n9XsDSNps/default.jpg)
![Hindsight Experience Replay — Covered by Kristian Holsheimer](https://i.ytimg.com/vi/9Nl49LLV62w/default.jpg)
![June Melbourne MLAI Meetup](https://i.ytimg.com/vi/oRkdLYGLk00/default.jpg)
![Improving Generalization in Deep Reinforcement Learning via SLC Weightings](https://i.ytimg.com/vi/iy3_Lqv9mPg/default.jpg)
![Exploring the Universe with Machine Learning - Prakrut Chaubal](https://i.ytimg.com/vi/rBB4XjkmA4g/default.jpg)