End to End AWS Data Engineering Project | Real time streaming.

𝗜 𝗵𝗮𝘃𝗲 𝗽𝗿𝗲𝗽𝗮𝗿𝗲𝗱 𝗖𝗼𝗺𝗽𝗹𝗲𝘁𝗲 𝗜𝗻𝘁𝗲𝗿𝘃𝗶𝗲𝘄 𝗣𝗿𝗲𝗽𝗮𝗿𝗮𝘁𝗶𝗼𝗻 𝗚𝘂𝗶𝗱𝗲 𝗳𝗼𝗿 𝗗𝗮𝘁𝗮 𝗘𝗻𝗴𝗶𝗻𝗻𝗲𝗿𝘀.

𝗚𝗲𝘁 𝘁𝗵𝗲 𝗚𝘂𝗶𝗱𝗲 𝗵𝗲𝗿𝗲 - https://lnkd.in/gJWSVKNM
𝗖𝗼𝘂𝗽𝗼𝗻 𝗰𝗼𝗱𝗲 - AZURE20 / AWS20 / GCP20 to get discounts for next 10 users.
Get End to End Data Engineering Project here: https://topmate.io/shubham_wadekar/1601125
Use Code PROJECT10 to get 10% discount.
In this video, you will understand the END TO END AWS DATA ENGINEERING PROJECT.
If you are someone who wants to understand how Data Engineering projects work in real life then this video is for you!
Connect with me:

Instagram: instagram.com/shubhaam.codes
LinkedIn: https://www.linkedin.com/in/shubhamwadekar/
Data Engineers - Interview Prep : https://linktr.ee/shubham_wadekar1
Whatsapp community: https://whatsapp.com/channel/0029VanKrG4DOQId4uezM11k
Interviewer: Chandrali Sarkar - https://www.linkedin.com/in/chandrali-sarkar-4570a1102/
Interviewee: Abhishek Mohanty - https://www.linkedin.com/in/abhishekmohanty3979/
CHAPTERS:

00:00 – Introduction of the Project Series (New Section)
01:04 – Introduction of Interviewer
01:40 - Introduction of the candidate
02:00 - Project Explanation
03:25 - How do you bring the data to aws from oracle
04:15 - How is Kafka integrated with the project
07:00 - Where does S3 fit in the architecture.
08:35 - Type of details present inside the data
11:00 - How is API invoked?
15:30 - Roles and responsibilities in the project.
17:00 - Monitoring tools
19:00 - Glue vs EMR
22:00 - Spark Jobs
23:25 - Spark batch vs spark streaming
24:00 - How much daily data you are handling on a day to day basis.
25:00 - Challenges faced
29:30 - Spark SQL vs dataframe vs dataset
32:10 - Quality checks
33:40 - How often things go down in real time
35.50 - Optimizations
38:50 - Why do you use parquet format?
40:00 - All about Athena
44:40 - Glue Crawler
46:50 - Key take aways from project and feedback
✨ Tags ✨
Mock Data Engineer Interview, Data Engineer Interview Questions, Data Engineering Mock Interview, Data Engineer Interview Preparation, Azure Data Engineer Interview, Cloud Data Engineering, ETL Interview Questions, SQL Interview for Data Engineer, PySpark Interview Questions, Big Data Interview Mock, Real Interview Data Engineer, Data Engineer Scenario Based Questions, Data Engineer Technical Interview, Mock Interview Full Length, AWS Data Engineer Mock, Data Engineering Projects, Data Engineer Career Switch, Entry Level Data Engineer, Data Engineering Roadmap, Data Modeling, Data warehousing, delta lake, glue, athena, stock market, real time data, Data Engineering project, Data Engineering projects, End to end data engineering project, GCP services, AWS Services
✨ HashTags ✨
#mockinterview #dataengineering #data #dataengineer #sql #python #pyspark #jobs #interview #bigdata #azure #cloud #gcp #aws #databricks #adf #azuredatafactory #synapse #dataflow #dataproc #project #end_to_end_data_engineering_project #ETL#DataPipeline #DataOps #PySpark #ApacheKafka #AWSGlue #Airflow #Snowflake #SQL #Python #CloudComputing #DataProcessing #BatchProcessing #StreamProcessing #DataTransformation #GoogleCloud #GCP #GoogleCloudPlatform #GCPProjects #CloudComputing #BigQuery #Dataflow #PubSub #Dataproc #CloudComposer #DataEngineeringOnGCP #GCPDataServices #StreamingWithGCP #CloudEngineer #GCPTraining #GCPforDataEngineers #aws #athena #glue #kafka #airflow

Видео End to End AWS Data Engineering Project | Real time streaming. канала Shubham Wadekar