Scalable and Reliable Logging at Pinterest (DataEngConf SF16)
In this talk, Krishna Gade (Head of Data Engineering) and Yu Yang (Data Engineer) will present Pinterest’s logging pipeline and share their experience addressing these challenges. They dive deep into three components they developed: data uploading from service hosts to Kafka, data transportation from Kafka to S3, and data sanitization. They also share their experience in operating Kafka at scale in the cloud.
ABOUT DATA COUNCIL:
Data Council (https://www.datacouncil.ai/) is a community and conference series that provides data professionals with the learning and networking opportunities they need to grow their careers. Make sure to subscribe to our channel for more videos, including DC_THURS, our series of live online interviews with leading data professionals from top open source projects and startups.
FOLLOW DATA COUNCIL:
Twitter: https://twitter.com/DataCouncilAI
LinkedIn: https://www.linkedin.com/company/datacouncil-ai
Facebook: https://www.facebook.com/datacouncilai
Eventbrite: https://www.eventbrite.com/o/data-council-30357384520
Видео Scalable and Reliable Logging at Pinterest (DataEngConf SF16) канала Data Council
ABOUT DATA COUNCIL:
Data Council (https://www.datacouncil.ai/) is a community and conference series that provides data professionals with the learning and networking opportunities they need to grow their careers. Make sure to subscribe to our channel for more videos, including DC_THURS, our series of live online interviews with leading data professionals from top open source projects and startups.
FOLLOW DATA COUNCIL:
Twitter: https://twitter.com/DataCouncilAI
LinkedIn: https://www.linkedin.com/company/datacouncil-ai
Facebook: https://www.facebook.com/datacouncilai
Eventbrite: https://www.eventbrite.com/o/data-council-30357384520
Видео Scalable and Reliable Logging at Pinterest (DataEngConf SF16) канала Data Council
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
Centralized Logging Patterns - Philipp KrennSystem Design Interview - Distributed CacheJosh Wills Keynote @ DataEngConf SF16Intro + Deep Dive: Prometheus - Julius Volz, Prometheus & Richard Hartmann, SpaceNetBuilding highly reliable data pipelines at Datadog | DatadogScaling Instagram InfrastructureHow load balancers work - System Design Interview knowledge [Beyond the interview]DevOps Over Coffee - AdidasBuilding a reliable and scalable metrics aggregation and monitoring system
by Vishnu GajendranScaling Slack - The Good, the Unexpected, and the Road AheadSystem Design Interview - Top K Problem (Heavy Hitters)Dropbox system design | Google drive system design | System design file share and uploadCodesmith Speaker Event: Google SRE - Designing Large Scale Distributed Systems [w/ Brett Beekley]System Design Interview - Distributed Message QueueScaling Pinterest • Marty Weiner • GOTO 2014System Design Interview - Notification ServiceHow Netflix Is Solving Authorization Across Their Cloud [I] - Manish Mehta & Torin Sandall, NetflixSystem Design Interview - Rate Limiting (local and distributed)F4 - Photo Storage at FacebookHow Superset and Druid Power Real-Time Analytics at Airbnb | DataEngConf SF '17