Fine-Tuned LLM Output Demo (Runs Fully Locally, No Cloud)

This video demonstrates the output of a fine-tuned LLM that I trained completely locally.

The model is capable of answering questions based on a dataset generated from a PDF document.

🔹 What this project involved (behind the scenes):
📄 PDF → Dataset → Fine-tuning → GGUF → Local Inference

• Generated a QA dataset using Ollama with Gemma 4B (fully local, no APIs)
• Fine-tuned Qwen3.5-0.8B using Unsloth
• Resolved issues like output looping, formatting errors, and parameter tuning
• Exported the model to GGUF format (Q5_K_M)
• Ran the model locally using a custom Modelfile via Ollama

🔹 What you're seeing in this video:
• Final model output after fine-tuning
• Example responses generated by the model
• Behavior of the model during local inference

🔹 Key Learnings:
• Model selection across the pipeline
• Effects of quantization on performance
• Fine-tuning stability and parameter tuning
• Importance of formatting during inference

🔹 Tools Used:
Unsloth · Qwen3.5-0.8B · Ollama · Gemma 4B · GGUF · Q5_K_M

Everything was executed locally — no cloud, no API costs.

#LLM #FineTuning #Ollama #LocalAI #MachineLearning #AIProjects

Видео Fine-Tuned LLM Output Demo (Runs Fully Locally, No Cloud) канала Pratham Desai