Data Engineering Course For Beginners - #2 TRANSFORM
This is the second part of the Free Data Engineering Course for Beginners that I've decided to create for you! Over the course of the four videos, we are going to cover the entire ETL process (extract, transform, load), and at the end we are also going to talk about job scheduling.
In this course you will build your first data feed (or data pipeline) using Spotify API. This feed will run daily, and it will download the data about the songs that you listened to during a day, and save that data in a SQLite database on your local machine.
In this video we are going to cover the Transform stage of the ETL process, which means that we will be learning how to validate the data that we received from a data vendor (Spotify in this case). We'll check for empty files, null values, stale data and duplicates! Along the way I will also explain some basic data engineering concepts such as a primary key constraint, or "garbage in, garbage out" principle.
Follow this link to generate your Spotify API token:
https://developer.spotify.com/console/get-recently-played/
Find the code with this data engineering project on GitHub:
https://github.com/karolina-sowinska/free-data-engineering-course-for-beginners/blob/master/main.py
Music:
What Now - Golden Age Radio
Connect with me on Instagram:
@karo_sowinska
And if you want to make my day with a cup of coffee... :)
https://ko-fi.com/karolina_sowinska
Видео Data Engineering Course For Beginners - #2 TRANSFORM канала Karolina Sowinska
In this course you will build your first data feed (or data pipeline) using Spotify API. This feed will run daily, and it will download the data about the songs that you listened to during a day, and save that data in a SQLite database on your local machine.
In this video we are going to cover the Transform stage of the ETL process, which means that we will be learning how to validate the data that we received from a data vendor (Spotify in this case). We'll check for empty files, null values, stale data and duplicates! Along the way I will also explain some basic data engineering concepts such as a primary key constraint, or "garbage in, garbage out" principle.
Follow this link to generate your Spotify API token:
https://developer.spotify.com/console/get-recently-played/
Find the code with this data engineering project on GitHub:
https://github.com/karolina-sowinska/free-data-engineering-course-for-beginners/blob/master/main.py
Music:
What Now - Golden Age Radio
Connect with me on Instagram:
@karo_sowinska
And if you want to make my day with a cup of coffee... :)
https://ko-fi.com/karolina_sowinska
Видео Data Engineering Course For Beginners - #2 TRANSFORM канала Karolina Sowinska
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
![Data Engineering Course For Beginners - #3 LOAD](https://i.ytimg.com/vi/rvPtpOjzVTQ/default.jpg)
![](https://i.ytimg.com/vi/XsUfEvmw4eo/default.jpg)
![How to Become a Data Engineer](https://i.ytimg.com/vi/q59rbUyhKCg/default.jpg)
![Data Engineering Course for Beginners - #1 EXTRACT](https://i.ytimg.com/vi/dvviIUKwH7o/default.jpg)
![Landing an ENTRY-LEVEL job in Machine Learning | Attack the ML job market LIKE A SNIPER](https://i.ytimg.com/vi/DGpdamK2o4A/default.jpg)
![Being A Data Engineer: Expectations vs Reality](https://i.ytimg.com/vi/6RiA_Qur2yo/default.jpg)
![Top 5 Career Paths for Data Professionals: Machine Learning & Machine Learning Engineering](https://i.ytimg.com/vi/gDg-tUACuvQ/default.jpg)
![What do DATA ENGINEERS do? Is data engineering a good career choice in 2020?](https://i.ytimg.com/vi/73QC3qw5b2Y/default.jpg)
![If you googled DOCKER but you still don't get it...](https://i.ytimg.com/vi/WkGV9rJJk3I/default.jpg)
![Airflow for Beginners - Run Spotify ETL Job in 15 minutes!](https://i.ytimg.com/vi/i25ttd32-eo/default.jpg)
![How to become a Data Engineer in 2021 FOR FREE!!!](https://i.ytimg.com/vi/ceOSlyc2J0g/default.jpg)
![Machine Learning vs Data Engineering - What's the difference?](https://i.ytimg.com/vi/cbsEWplQH0k/default.jpg)
![The Art of Winning an Argument: 32 DIRTY TRICKS of Schopenhauer - Part 1](https://i.ytimg.com/vi/CanelJD8WYc/default.jpg)
![Functional Data Engineering - A Set of Best Practices | Lyft](https://i.ytimg.com/vi/4Spo2QRTz1k/default.jpg)
![Why NOT to become a Data Engineer](https://i.ytimg.com/vi/UjYc8uH6lHw/default.jpg)
![How Data Engineering Works](https://i.ytimg.com/vi/qWru-b6m030/default.jpg)
![Confessions from a Big Tech Hiring Manager: Tips for Software Engineering Interviews](https://i.ytimg.com/vi/vFOw_m5zNCs/default.jpg)
![SOFTWARE JOB IN GERMANY | SALARY IN GERMANY | German Work Culture | My Interview Experience](https://i.ytimg.com/vi/s__sY35_8io/default.jpg)
![Don't ever write Python code like this](https://i.ytimg.com/vi/--_K4G3HCcI/default.jpg)
![How to switch your career to coding (from Economics to Data Science/Software Engineering)](https://i.ytimg.com/vi/ZBfPyTCF5xw/default.jpg)