Kubernetes Logs ➡️ FluentBit ➡️ Grafana Loki (The Easy Way!) #observability #grafana #logging

In this video, we dive deep into Kubernetes logging and walk through how to build an efficient log data pipeline. You'll learn the core concepts of Fluent Bit, what a data pipeline actually is, and how to seamlessly send your K8s cluster logs to Grafana Loki for centralized monitoring.
Whether you are a DevOps engineer or just getting started with cloud-native observability, this step-by-step tutorial will help you set up production-ready logging in minutes!

🚀 What we cover in this video:
00:00 - Introduction & Recap of Previous Grafana Loki Deployment Modes
00:27 - When to Use Microservices vs. Single Binary Deployment Modes
00:59 - Overview of Today's Demo Setup (Multi-Cluster Logging Flow)
01:31 - Alternative Logging Agents (CloudWatch, OpenSearch, Splunk)
01:59 - What is a Data Pipeline in Fluent Bit?
02:42 - Fluent Bit Core Concepts: Inputs & Plugins (Tail vs. HTTP)
03:45 - Fluent Bit Core Concepts: Processors (Masking Sensitive/PII Data)
04:39 - Fluent Bit Core Concepts: Parsers (Structuring Unstructured Logs)
05:12 - Fluent Bit Core Concepts: Filters (Enriching Logs with Metadata)
06:21 - Fluent Bit Core Concepts: Buffers (Memory vs. File System Storage)
06:53 - Why Do We Need Buffers? (Handling Unresponsive Destinations)
07:20 - Fluent Bit Core Concepts: Routing Policies (Sending to S3, Loki, OpenSearch)
08:09 - Official Fluent Bit Data Pipeline Documentation Walkthrough
08:52 - Deprecation of Classic .conf Files & Moving to YAML (YAML Config Setup)
09:33 - Local Home Lab Lab Architecture (Virtual Machines, Docker Desktop & K8s)
11:07 - Summary of the 3 Hands-On Blueprint Steps
11:47 - Step 1: Connecting Grafana to the Loki Data Source Container
12:47 - Step 2: Deploying the Mock Application Shell Script in Kubernetes (app & app2 Namespaces)
14:38 - Verifying Mock App Container Logs via kubectl
15:05 - Step 3: Deploying Fluent Bit via Helm Chart using Custom values.yaml
16:45 - Bare Minimum Pipeline Setup (Tail Input Plugin to Loki Output)
18:34 - Executing the Helm Upgrade/Install and Verifying DaemonSet Pods
20:01 - Visualizing the Initial Logs in Grafana Explore Panel
20:40 - Activating the Kubernetes Filter Plugin for Rich Metadata
21:36 - Verifying Enriched Log Fields (Namespaces, Pod IPs, Container Names) in Grafana
22:16 - Using the Modify Filter to Clean Logs (Removing stream and _p keys)
23:25 - Enhancing Loki Output Labels for Query Indexing
24:32 - Advanced Log Filtering in Grafana Using Indexed Labels (Filter by Namespace, Container & Error Level)
25:31 - Why Label Indexing is Critical in Grafana Loki vs. Log Messages
25:52 - Outro & Closing Remarks

🔗 Resources & Reference Links:
GitHub Repository: https://github.com/deploymasters/youtube/tree/main/observability/fluentbit
Fluent Bit Official Documentation: https://docs.fluentbit.io/
Grafana Loki Documentation: https://grafana.com/docs/loki/latest/
Kubernetes Logging Architecture: https://kubernetes.io/docs/concepts/cluster-administration/logging/

🤝 Reach Out:
Have questions, feedback, or want to collaborate? Feel free to reach out to me here:
LinkedIn: https://www.linkedin.com/in/iambalvinders/
Telegram: https://t.me/deploymasters
Discord: https://discord.gg/8hXsCG366j

#kubernetes #fluentbit #grafana #grafanaloki #devops #cloudnative #logging #observability

Видео Kubernetes Logs ➡️ FluentBit ➡️ Grafana Loki (The Easy Way!) #observability #grafana #logging канала Deploy Masters