Langfuse Goes Fully Open Source: Self Hosting & LLM as a Judge Walkthrough

🚀 LangFuse just went FULLY open source! In this comprehensive walkthrough, we dive deep into their Launch Week 3 announcements and show you how to self-host Langfuse with their new Terraform modules, plus demo the complete evaluation workflow with LLM as a Judge.

What we cover in this tutorial:
- Langfuse's massive open source announcement (all pro features now free!)
- Real-world experience with new Terraform modules for AWS, GCP & Azure
- Self-hosting setup with Helm charts and Kubernetes
- Complete AI evaluation pipeline using a recipe chatbot
- LLM as a Judge implementation with strict vs lenient evaluators
- Annotation queues for human evaluation
- Prompt experiments and dataset management
- What worked (and what didn't) in our deployment

🔗 Key Resources:
- Langfuse Announcement: https://langfuse.com/blog/2025-06-04-open-sourcing-langfuse-product
- Langfuse Self-Hosting Docs: https://langfuse.com/self-hosting
- Terraform Modules: https://github.com/langfuse/langfuse-terraform-aws
- Recipe Bot: https://github.com/ai-evals-course/recipe-chatbot
- Recipe Bot Fork: https://github.com/app-vitals/recipe-chatbot
- Langfuse Evaluator Library: https://langfuse.com/docs/scores/model-based-evals

👋🏻 About Us:
Hi! We're Dan and Dave, the founders of App Vitals. On this channel, we share practical tutorials that teach developers how to build production-ready AI systems that actually work in the real world.

🗓 Need help implementing AI observability in your business? Book a free consultation:
https://cal.com/dan-mcaulay/discovery-call

💬 COMMENT BELOW: Are you planning to self-host LangFuse now that it's fully open source? What evaluation challenges are you facing?

Видео Langfuse Goes Fully Open Source: Self Hosting & LLM as a Judge Walkthrough канала App Vitals