Airflow for Beginners - Run Spotify ETL Job in 15 minutes!
In this long-awaited Airflow for Beginners video I'm showing you how to install Airflow from scratch, and how to schedule your first ETL job in Airflow!
We will run the Spotify ETL job that we created in my Data Engineering Course for Beginners. But if you have your own code to schedule in Airflow - feel free to use it instead!
In this Airflow tutorial I just wanted to get you started with Airflow, because I know that configuring Airflow can be non-trivial. I won't be going in-depth into all Airflow's features, so hopefully this tutorial will be as beginners friendly as it can get!
I'll show you how to fix some installation errors, how to set up your project folders, how to edit Airflow config file and where it sits, and how to write your first DAG file!
Get ready for a ride!
Oh, but grab some useful links first:
-github code repo: https://github.com/karolina-sowinska/free-data-engineering-course-for-beginners
-Airflow quick start guide: https://airflow.apache.org/docs/stable/start.html
-Spotify access token: https://developer.spotify.com/console/get-recently-played/
Don't forget to subscribe if you're new here, more content is coming!
:)
Find me on Instagram:
@karo_sowinska
Видео Airflow for Beginners - Run Spotify ETL Job in 15 minutes! канала Karolina Sowinska
We will run the Spotify ETL job that we created in my Data Engineering Course for Beginners. But if you have your own code to schedule in Airflow - feel free to use it instead!
In this Airflow tutorial I just wanted to get you started with Airflow, because I know that configuring Airflow can be non-trivial. I won't be going in-depth into all Airflow's features, so hopefully this tutorial will be as beginners friendly as it can get!
I'll show you how to fix some installation errors, how to set up your project folders, how to edit Airflow config file and where it sits, and how to write your first DAG file!
Get ready for a ride!
Oh, but grab some useful links first:
-github code repo: https://github.com/karolina-sowinska/free-data-engineering-course-for-beginners
-Airflow quick start guide: https://airflow.apache.org/docs/stable/start.html
-Spotify access token: https://developer.spotify.com/console/get-recently-played/
Don't forget to subscribe if you're new here, more content is coming!
:)
Find me on Instagram:
@karo_sowinska
Видео Airflow for Beginners - Run Spotify ETL Job in 15 minutes! канала Karolina Sowinska
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
Data Engineering Course for Beginners - #1 EXTRACTAirflow tutorial 2: Set up airflow environment with dockerHow to switch your career to coding (from Economics to Data Science/Software Engineering)What do DATA ENGINEERS do? Is data engineering a good career choice in 2020?Limits to introspectionAirflow tutorial 4: Writing your first pipelineAirflow: Automating ETLs for a Data Warehouse, Natarajan Chakrapani, SF Python July 2018Amazon Managed Workflows for Apache Airflow: Getting StartedLa conquête de nos workflows avec Apache Airflow - Germain Tanguy & Cedric HourcadeAirflow tutorial 1: Introduction to Apache AirflowData Engineering Principles - Build frameworks not pipelines - Gatis SejaHow to Become a Data Engineer[Getting started with Airflow - 6] Using sensors to sense filesSoftware Engineering or Machine Learning: What's a better career?Ingest, prepare & transform using Azure Databricks & Data Factory | Azure FridayMachine Learning vs Data Engineering - What's the difference?I show 16-inch MacBook Pro's real-world performance on MACHINE LEARNING algorithmsAirflow 2.0: What to expect?Going with the flow: Intro to Airflow[Getting started with Airflow - 1] Installing and running Airflow using docker and docker-compose