How To Create Data Pipeline on AWS? || Techtter ||
In this video I shared my knowledge on how to build Data Pipeline on AWS (Amazon Webservices) cloud using available tools like Kinesis / Apache Kafka, Lambda function, Glue (serverless ETL pipeline), EMR Cluster, Redshift, Metabase and Quicksight for visualization etc.,
⏱️TIMESTAMPS ⏱️
0:00 - Introduction
0:54 - Challenges while building Data Pipeline
3:19 - Data Pipeline Design
3:28 - Data Sources
3:58 - Data Ingestion
5:06 - Data Storage with Amazon S3 / ETL with Glue / EMR
8:09 - Data Warehousing
8:50 - Data Visualization
📌 Related Videos:
► Spark Streaming Vs Kafka Streams: https://youtu.be/0ckzV7JwPnQ
#aws #datapipeline #techtter
----------------------------------------------------------------------------------------------
✅ Subscribe To Channel on Social Media:
YOUTUBE: ► https://www.youtube.com/Techtter?sub_confirmation=1
FACEBOOK: ► https://www.facebook.com/Techtter/
TWITTER: ► https://www.twitter.com/TheTechtter
INSTAGRAM: ► https://www.instagram.com/Techtter
DEV: ► https://dev.to/techtter
WEB: ► https://www.techtter.net
-----------------------------------------------------------------------------------------------
🔥 Disclaimer/Policy:
The content of the videos, ideas given in videos all are self opinionated. The code samples shown in the video are collected from opensource projects shared on github and from other sources. The trademarks and logos shown on thumbnails and in videos are registered for respective companies. This video is not certified or accredited by them.
Видео How To Create Data Pipeline on AWS? || Techtter || канала Techtter
⏱️TIMESTAMPS ⏱️
0:00 - Introduction
0:54 - Challenges while building Data Pipeline
3:19 - Data Pipeline Design
3:28 - Data Sources
3:58 - Data Ingestion
5:06 - Data Storage with Amazon S3 / ETL with Glue / EMR
8:09 - Data Warehousing
8:50 - Data Visualization
📌 Related Videos:
► Spark Streaming Vs Kafka Streams: https://youtu.be/0ckzV7JwPnQ
#aws #datapipeline #techtter
----------------------------------------------------------------------------------------------
✅ Subscribe To Channel on Social Media:
YOUTUBE: ► https://www.youtube.com/Techtter?sub_confirmation=1
FACEBOOK: ► https://www.facebook.com/Techtter/
TWITTER: ► https://www.twitter.com/TheTechtter
INSTAGRAM: ► https://www.instagram.com/Techtter
DEV: ► https://dev.to/techtter
WEB: ► https://www.techtter.net
-----------------------------------------------------------------------------------------------
🔥 Disclaimer/Policy:
The content of the videos, ideas given in videos all are self opinionated. The code samples shown in the video are collected from opensource projects shared on github and from other sources. The trademarks and logos shown on thumbnails and in videos are registered for respective companies. This video is not certified or accredited by them.
Видео How To Create Data Pipeline on AWS? || Techtter || канала Techtter
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
AWS Data Pipeline Tutorial | AWS Tutorial For Beginners | AWS Certification Training | EdurekaBuilding data pipelines with Kafka and PostgreSQLData Engineering Principles - Build frameworks not pipelines - Gatis SejaBatch Processing Example using Amazon Data Pipeline | S3 to DynamoDB using Amazon EMR | Tech PrimersStages of Pipeline ConstructionAWS Glue | Importing CSV files from S3 into RedshiftWhat is Data Pipeline | How to design Data Pipeline ? - ETL vs Data pipelineHadoop on AWS using EMR Tutorial || S3 || Athena || Glue || QuickSightBest Practices for Building a Cloud Data PipelineDesigning the Next Generation of Data Pipelines at Zillow with Apache SparkAWS Tutorials - Create Data Lake with Amazon S3, Lake Formation and GlueAWS Glue Tutorial | Getting Started with AWS Glue ETL | AWS Tutorial for Beginners | EdurekaBuilding Serverless Data Analytics Pipeline on AWS - AWS Online Tech TalksCreate a Data Pipeline, Lake, and Warehouse using AWSData Pipelines with Python and PostgreSQLBuilding Your Data Lake on AWSSchedule EMR Job using AWS DataPipeline - Explained Using SparkPart 1 - A sample data pipeline using Python - Setup Project and Develop Program to read table listAWS Quicksight - Build Dashboard TutorialBuild ETL Processes for Data Lakes with AWS Glue - AWS Online Tech Talks