Build an End-to-End ETL Pipeline Using open source stack; MinIO, Airbyte, dbt, and Postgres

In this video we are building a complete end to end ETL pipeline with open source data stack. No managed services or vendor lock-in. We will use MinIO as our data source. It’s an S3 compatible service. We will ingest data from MinIO with the help of Airbyte and load it into a Postgres database. Using dbt we will transform this data into a final reporting layer as dimensions and facts. We will use Airflow to orchestrate the whole ETL pipeline.
By the end, you'll have a fully functional ETL pipeline running locally.

GitHub repo:
https://github.com/hnawaz007/modern-data-stack-etl-

MinIO Install on docker video:
https://www.youtube.com/watch?v=DLRiUs1EvhM

Airbyte Install video:
https://www.youtube.com/watch?v=Ryr97YYlARY

Airflow Installation video:
https://www.youtube.com/watch?v=In7zwp0FDX4

Postgres install video:
https://www.youtube.com/watch?v=fjYiWXHI7Mo

Python Install video:
https://www.youtube.com/watch?v=B0G-44dqHRM

#etl #dataengineering #moderndatastack

💥Subscribe to our channel:
https://www.youtube.com/c/HaqNawaz

📌 Links
-----------------------------------------
#️⃣ Follow me on social media! #️⃣

🔗 GitHub: https://github.com/hnawaz007
📸 Instagram: https://www.instagram.com/bi_insights_inc
📝 LinkedIn: https://www.linkedin.com/in/haq-nawaz/
🔗 https://medium.com/@hnawaz100

Видео Build an End-to-End ETL Pipeline Using open source stack; MinIO, Airbyte, dbt, and Postgres канала BI Insights Inc