Day 11 of Data Engineerinc Zoomcamp || Orchestrating dbt Model w Postgres in Kestra #dataengineering

📌 Data Engineering Zoomcamp 🚀

🗓 Day 11 | ETL with Kestra Part 3 - Orchestrating dbt Models with Postgres

🔍 Today's Topics:
✅ Running dbt in Kestra
✅ Transforming raw Postgres data with dbt models
✅ Defining dbt models directly from Kestra UI using simple YAML

📖 Key Takeaways:
📝 After ingesting data into Postgres and setting up scheduling & backfills, we now use dbt to clean, structure, and make the data analytics-ready.
💡 Kestra simplifies dbt orchestration—just a few lines of YAML to define and trigger models from the UI, no extra setup needed!

🔍 dbt Models Output:
✨ Raw Taxi Trip Data → Transformed Tables
📊 Standardized columns, cleaned nulls
📍 Derived metrics (trip duration, fare calculations)
📌 Ready for analytics & reporting

👨‍💻 Hands-on Practice:
🔹 Created a dbt workflow in Kestra
🔹 Defined & triggered dbt models from the UI
🔹 Transformed and validated data in Postgres

📢 Thoughts:
💬 Orchestrating dbt models with Kestra is incredibly smooth! The ability to define transformations directly in YAML and run them seamlessly is a game-changer. Now the data is structured, enriched, and ready for insights!

👤 About Me:
Hi, I’m Jo, a BI Engineer passionate about data, automation, and problem-solving. I’m currently on a 6-week journey to upskill in data engineering through DE Zoomcamp 2025 by DataTalks.Club. Follow along as I share my daily learnings! 🚀

📌 Follow my journey: #dailyincremental | #dataengineeringzoomcamp2025

#dataengineering #etl #kestra #postgres #dbt #workfloworchestration #bigdata #analyticsengineering #sql #dataautomation

Видео Day 11 of Data Engineerinc Zoomcamp || Orchestrating dbt Model w Postgres in Kestra #dataengineering канала Daily Incremental