📉 Calculating Text Generation Loss – Live Coding with Sebastian Raschka (Chapter 5.1.2)

Check out Sebastian Raschka's book 📖 Build a Large Language Model (From Scratch) | https://hubs.la/Q03l0mSf0 📖

In this essential live-coding session, LLM expert @SebastianRaschka covers Chapter 5.1.2: Calculating the Text Generation Loss from his book Build a Large Language Model (From Scratch). Learn how to compute the loss function that drives training for GPT-style language models — enabling the model to improve its ability to predict the next token in a sequence.

0:00 - Introduction to Text Generation Loss
0:50 - Explaining Cross Entropy Loss
1:42 - Understanding Inputs, Targets, and Shifting
2:29 - Practical Example: Logits and Probabilities
8:13 - Calculating Probabilities for Target Tokens
13:00 - Extracting Probabilities for Targets
16:26 - Logarithms, Average Log-Probability, and Cross Entropy
22:03 - Using PyTorch's Cross Entropy Function
26:10 - Training Objective Recap and Next Steps

📘 About the Book
Build a Large Language Model (From Scratch) is a practical and eminently-satisfying hands-on journey into the foundations of generative AI. Without relying on any existing LLM libraries, you’ll code a base model, evolve it into a text classifier, and ultimately create a chatbot that can follow your conversational instructions. And you’ll really understand it because you built it yourself!

💡 Ideal for ML engineers, researchers, and developers seeking to build and understand large-scale text generation models.

🔗 Get the Book: https://hubs.la/Q03l0mSf0

📺 Subscribe to Manning Publications for expert tutorials, live coding walkthroughs, and deep dives into transformer training workflows.

#SebastianRaschka #TextGeneration #LanguageModeling #GPT #CrossEntropyLoss #LLM #Transformers #PyTorch #MachineLearning #NLP #ManningPublications #LiveCoding

Видео 📉 Calculating Text Generation Loss – Live Coding with Sebastian Raschka (Chapter 5.1.2) канала Manning Publications