Build an LLM from Scratch 6: Finetuning for Classification

Links to the book:
- https://amzn.to/4fqvn0D (Amazon)
- https://mng.bz/M96o (Manning)

Link to the GitHub repository: https://github.com/rasbt/LLMs-from-scratch

This is a supplementary video explaining how to finetune an LLM as a classifier (here using a spam classification example) as a gentle introduction to fine-tuning, before instruction finetuning the LLM in the next video.

00:00 6.2 Preparing the dataset
26:49 6.3 Creating data loaders
42:50 6.4 Initializing a model with pretrained weights
52:56 6.5 Adding a classification head
1:08:28 6.6 Calculating the classification loss and accuracy
1:30:54 6.7 Finetuning the model on supervised data
2:04:25 6.8 Using the LLM as a spam classifier

You can find additional bonus materials on GitHub:

Additional experiments finetuning different layers and using larger models,
https://github.com/rasbt/LLMs-from-scratch/tree/main/ch06/02_bonus_additional-experiments

Finetuning different models on 50k IMDB movie review dataset,
https://github.com/rasbt/LLMs-from-scratch/tree/main/ch06/03_bonus_imdb-classification

Building a User Interface to Interact With the GPT-based Spam Classifier,
https://github.com/rasbt/LLMs-from-scratch/tree/main/ch06/04_user_interface

Видео Build an LLM from Scratch 6: Finetuning for Classification канала Sebastian Raschka