Chapter #9 - How to design data pipeline on gcp (Google Cloud Platform) ?
Chapter #9 - Designing data pipeline solution on GCP
#datapipeline #googlecloud #gcp
What is Data Pipeline? - https://youtu.be/PjBhIDOdLM0
***Google Cloud Platform Beginner Series***
Google Cloud Platform Beginner Series (2020) - https://www.youtube.com/watch?v=OzwSBbuHY-0&list=PLcnJIHtHiTA2Jp3klIkeHiwqGrxjXBzh6
***Networking Basics Playlist***
Networking and Infra Concepts - https://www.youtube.com/playlist?list=PLcnJIHtHiTA0C9UXgU2C6eTQMVFzSlIdP
*** Other Popular Playlists***
Crunching Data Series (2020) - https://www.youtube.com/playlist?list=PLcnJIHtHiTA2HmIleev4scARSFwrQ0bIy
Latest technology tutorial (2020) -
https://www.youtube.com/watch?v=l5UcUEt1IzM&list=PLcnJIHtHiTA2Hdgon51_-G3Kt43hcU6kI
Hi Friends, I am Anshul Tiwari, and welcome to our youtube channel IT k Funde.
More about this video -
In this video, we will learn something different, instead of discussing any specific product on google cloud platform, we will learn how to design a data pipeline solution on google cloud platform.
This will help us in 2 ways -
1. This will helps us improve our design thinking
2. We will get an overview of all google cloud products that can be used in developing a data pipeline.
We will start with understanding a typical on premise data pipeline and enterprise data warehouse (edw) design. We will then take this same solution and try to implement it on google cloud platform.
On-Prem solution design contains below products -
#Streaming data - Apache Kafka
#ETL - Informatica, IBM Datastage, SAP Data Services
#EDW - Teradata
#Dashbaords & reporting - Tableau and SAP Business Objects
GCP cloud design contains below products -
#Streaming data - Cloud PubSub
#ETL - Cloud Data Fusion
#Storage - Google Cloud Storage
#EDW - Google Big Query
#Dashboards - Looker, Google Data Studio
#ML - AutoML
#Metadata Management - Data Catalog
PLEASE WATCH ALL THE VIDEOS IN ANY PLAYLIST FOR BETTER UNDERSTANDING !!
Credits & Resources for further study -
GCP resources documentation for detailed study -
#Cloud PubSub - https://cloud.google.com/pubsub/docs/overview
#Cloud Data Fusion - https://cloud.google.com/data-fusion/docs/concepts/overview
#Google Cloud Storage - https://cloud.google.com/storage/docs/quickstart-console
#Google Big Query- https://cloud.google.com/bigquery/docs/quickstarts
#Data Studio - https://cloud.google.com/bi-engine/docs/getting-started-data-studio
#Looker - https://docs.looker.com/
#AutoML - https://cloud.google.com/automl/docs
#Data Catalog - https://cloud.google.com/data-catalog/docs
images - pixabay.com
***Social Links***
Youtube - https://www.youtube.com/c/itkfunde
Facebook - https://www.facebook.com/ITkFUNDE/
Linkedin - https://www.linkedin.com/in/ansh9685
Twitter - https://twitter.com/ansh9685
Instagram -https://www.instagram.com/itkfunde/
***About This Channel***
Friends ITkFUNDE channel wants to bring I.T. industry knowledge, information, career advice, and much more to every individual regardless of whether he or she belongs to I.T or not. This channel is for everyone interested in learning something new!
Видео Chapter #9 - How to design data pipeline on gcp (Google Cloud Platform) ? канала IT k Funde
#datapipeline #googlecloud #gcp
What is Data Pipeline? - https://youtu.be/PjBhIDOdLM0
***Google Cloud Platform Beginner Series***
Google Cloud Platform Beginner Series (2020) - https://www.youtube.com/watch?v=OzwSBbuHY-0&list=PLcnJIHtHiTA2Jp3klIkeHiwqGrxjXBzh6
***Networking Basics Playlist***
Networking and Infra Concepts - https://www.youtube.com/playlist?list=PLcnJIHtHiTA0C9UXgU2C6eTQMVFzSlIdP
*** Other Popular Playlists***
Crunching Data Series (2020) - https://www.youtube.com/playlist?list=PLcnJIHtHiTA2HmIleev4scARSFwrQ0bIy
Latest technology tutorial (2020) -
https://www.youtube.com/watch?v=l5UcUEt1IzM&list=PLcnJIHtHiTA2Hdgon51_-G3Kt43hcU6kI
Hi Friends, I am Anshul Tiwari, and welcome to our youtube channel IT k Funde.
More about this video -
In this video, we will learn something different, instead of discussing any specific product on google cloud platform, we will learn how to design a data pipeline solution on google cloud platform.
This will help us in 2 ways -
1. This will helps us improve our design thinking
2. We will get an overview of all google cloud products that can be used in developing a data pipeline.
We will start with understanding a typical on premise data pipeline and enterprise data warehouse (edw) design. We will then take this same solution and try to implement it on google cloud platform.
On-Prem solution design contains below products -
#Streaming data - Apache Kafka
#ETL - Informatica, IBM Datastage, SAP Data Services
#EDW - Teradata
#Dashbaords & reporting - Tableau and SAP Business Objects
GCP cloud design contains below products -
#Streaming data - Cloud PubSub
#ETL - Cloud Data Fusion
#Storage - Google Cloud Storage
#EDW - Google Big Query
#Dashboards - Looker, Google Data Studio
#ML - AutoML
#Metadata Management - Data Catalog
PLEASE WATCH ALL THE VIDEOS IN ANY PLAYLIST FOR BETTER UNDERSTANDING !!
Credits & Resources for further study -
GCP resources documentation for detailed study -
#Cloud PubSub - https://cloud.google.com/pubsub/docs/overview
#Cloud Data Fusion - https://cloud.google.com/data-fusion/docs/concepts/overview
#Google Cloud Storage - https://cloud.google.com/storage/docs/quickstart-console
#Google Big Query- https://cloud.google.com/bigquery/docs/quickstarts
#Data Studio - https://cloud.google.com/bi-engine/docs/getting-started-data-studio
#Looker - https://docs.looker.com/
#AutoML - https://cloud.google.com/automl/docs
#Data Catalog - https://cloud.google.com/data-catalog/docs
images - pixabay.com
***Social Links***
Youtube - https://www.youtube.com/c/itkfunde
Facebook - https://www.facebook.com/ITkFUNDE/
Linkedin - https://www.linkedin.com/in/ansh9685
Twitter - https://twitter.com/ansh9685
Instagram -https://www.instagram.com/itkfunde/
***About This Channel***
Friends ITkFUNDE channel wants to bring I.T. industry knowledge, information, career advice, and much more to every individual regardless of whether he or she belongs to I.T or not. This channel is for everyone interested in learning something new!
Видео Chapter #9 - How to design data pipeline on gcp (Google Cloud Platform) ? канала IT k Funde
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
What is Data Pipeline | How to design Data Pipeline ? - ETL vs Data pipelineData Journey EP-02: Batch Ingestion 📦 - 5 ways to ingest files into Google CloudChapter #7 - Networking on cloud | What is vpc (virtual private cloud)- Google cloud platform basicsCloud Data Fusion: Data Integration at Google Cloud (Cloud Next '19)What is a Data Warehouse - Explained with real life example | datawarehouse vs database (2020)Lesson #4 - How to create docker container on google cloud platform (gcp) | Step by Step (2020)What is Data Lake (2019) | Data Lake vs Data Warehouse (English Subtitles)The Insane Biology of: The OctopusKubernetes crash course: In less than 15 minutesGoogle Cloud Dataflow5 Basic Networking commands for everyone (2021) | How to troubleshoot network issues on Windows?Artificial intelligence and algorithms: pros and cons | DW Documentary (AI documentary)Google Cloud Platform Tutorial | What is Google Cloud Platform | GCP Training | EdurekaData Warehousing With BigQuery: Best Practices (Cloud Next '19)What is nosql ? | sql vs nosql | Types of NOSQL databases - Explained with real life example (2020)Build ETL Pipelines using Cloud DataflowApache Kafka Basics - A Layman's guide for beginners | Explained with real life examplesDevelopment vs Support - How to know which is better for my career ? (dev or support)How a laptop get its IP ? What is DHCP | How dhcp works ?(Dynamic Host Configuration Protocol)