"Building a Distributed Task Scheduler With Akka, Kafka, and Cassandra" by David van Geest
Dynamically scheduled tasks are at the heart of PagerDuty's microservices. They deliver incident alerts, on-call notifications, and manage myriad administrative chores. Historically, these tasks were scheduled and run using an in-house library built on Cassandra, but that solution had begun to show its age.
Early in 2016, the Core team at PagerDuty built a new Task Scheduler using Akka, Kafka, and Cassandra. After six weeks in development, the Scheduler is now running in production. This talk discusses how the strengths of the three technologies were leveraged to solve the challenges of resilient, distributed task scheduling.
This talk will present a number of distributed system concepts in the real-world context of the Scheduler project. How can you dynamically adjust for increased task load with zero downtime? Can you guarantee task ordering across many servers? Do your tasks still run when an entire datacenter goes down? What happens if your tasks are scheduled twice? Attendees can expect to see how all of these challenges were addressed.
Some familiarity with distributed queueing and actor systems will be helpful for attendees of this talk.
Видео "Building a Distributed Task Scheduler With Akka, Kafka, and Cassandra" by David van Geest канала Strange Loop Conference
Early in 2016, the Core team at PagerDuty built a new Task Scheduler using Akka, Kafka, and Cassandra. After six weeks in development, the Scheduler is now running in production. This talk discusses how the strengths of the three technologies were leveraged to solve the challenges of resilient, distributed task scheduling.
This talk will present a number of distributed system concepts in the real-world context of the Scheduler project. How can you dynamically adjust for increased task load with zero downtime? Can you guarantee task ordering across many servers? Do your tasks still run when an entire datacenter goes down? What happens if your tasks are scheduled twice? Attendees can expect to see how all of these challenges were addressed.
Some familiarity with distributed queueing and actor systems will be helpful for attendees of this talk.
Видео "Building a Distributed Task Scheduler With Akka, Kafka, and Cassandra" by David van Geest канала Strange Loop Conference
Показать
Комментарии отсутствуют
Информация о видео
20 сентября 2016 г. 6:31:05
00:36:39
Другие видео канала
System Design Interview - Distributed Message QueueThe Basics of Enterprise Job Scheduling"We Really Don't Know How to Compute!" - Gerald Sussman (2011)Building resilient scheduling in distributed systems with Spring by Marek Jeszka @ Spring I/O 2019Spring Cloud Session-4 Inter Microservice Communication ASynchronous using KafkaScaling Instagram InfrastructureModern Data Architectures with Kafka and Cassandra | DataStax Accelerate 2019Scalable and Reliable Logging at Pinterest (DataEngConf SF16)FACEBOOK CODING INTERVIEW QUESTION - TASK SCHEDULER (LeetCode)Building Low Latency Trading SystemsSystem Design Interview - Top K Problem (Heavy Hitters)Role of Solution Architect in Software Development, Compared with Enterprise and Software ArchitectsZookeeper in action by Alexandre BerthaudFour Distributed Systems Architectural Patterns by Tim BerglundEBtree - Design for a Scheduler and Use (Almost) EverywhereBuilding Event-Driven Microservices with Event Sourcing and CQRS - Lidan HifiSystem Design Interview - Notification ServiceLessons learned form Kafka in production (Tim Berglund, Confluent)Design Tic Tac Toe: Low Level Design Coding Interview QuestionTangram Distributed Scheduling Framework for Apache Spark at Facebook