Kafka as a Platform: the Ecosystem from the Ground Up - Robin Moffatt - NDC Oslo 2020
Kafka has become a key data infrastructure technology, and we all have at least a vague sense that it is a messaging system, but what else is it? How can an overgrown message bus be getting this much buzz? Well, because Kafka is merely the center of a rich streaming data platform that invites detailed exploration.
In this talk, we’ll look at the entire streaming platform provided by Apache Kafka and the Confluent community components. Starting with a lonely key-value pair, we’ll build up topics, partitioning, replication, and low-level Producer and Consumer APIs. We’ll group consumers into elastically scalable, fault-tolerant application clusters, then layer on more sophisticated stream processing APIs like Kafka Streams and ksqlDB. We’ll help teams collaborate around data formats with schema management. We’ll integrate with legacy systems without writing custom code. By the time we’re done, the open-source project we thought was Big Data’s answer to message queues will have become an enterprise-grade streaming platform, all in 60 minutes.
Timecodes
0:00 Intro
01:24 What are Events?
04:33 A platform for events
05:11 The Log
06:54 Topics
08:53 Partitions
09:37 Publisher API
10:45 Message keys
14:10 Consumer API
19:20 Consumer Groups
22:30 Brokers
24:39 Demo 01 - Kafka producer/consumer
30:30 Kafka use cases
32:43 Kafka Connect
36:40 Schemas & Serialisation Formats
44:40 Stream processing introduction
46:50 Kafka Streams
48:03 ksqlDB
50:16 Demo 02
57:05 Recap & Resources
📚 Free eBooks: http://rmoff.dev/ndcoslo2020
✍️ Slides: https://rmoff.dev/kafka101
🎥 More Apache Kafka videos: https://www.youtube.com/rmoff
Видео Kafka as a Platform: the Ecosystem from the Ground Up - Robin Moffatt - NDC Oslo 2020 канала NDC Conferences
In this talk, we’ll look at the entire streaming platform provided by Apache Kafka and the Confluent community components. Starting with a lonely key-value pair, we’ll build up topics, partitioning, replication, and low-level Producer and Consumer APIs. We’ll group consumers into elastically scalable, fault-tolerant application clusters, then layer on more sophisticated stream processing APIs like Kafka Streams and ksqlDB. We’ll help teams collaborate around data formats with schema management. We’ll integrate with legacy systems without writing custom code. By the time we’re done, the open-source project we thought was Big Data’s answer to message queues will have become an enterprise-grade streaming platform, all in 60 minutes.
Timecodes
0:00 Intro
01:24 What are Events?
04:33 A platform for events
05:11 The Log
06:54 Topics
08:53 Partitions
09:37 Publisher API
10:45 Message keys
14:10 Consumer API
19:20 Consumer Groups
22:30 Brokers
24:39 Demo 01 - Kafka producer/consumer
30:30 Kafka use cases
32:43 Kafka Connect
36:40 Schemas & Serialisation Formats
44:40 Stream processing introduction
46:50 Kafka Streams
48:03 ksqlDB
50:16 Demo 02
57:05 Recap & Resources
📚 Free eBooks: http://rmoff.dev/ndcoslo2020
✍️ Slides: https://rmoff.dev/kafka101
🎥 More Apache Kafka videos: https://www.youtube.com/rmoff
Видео Kafka as a Platform: the Ecosystem from the Ground Up - Robin Moffatt - NDC Oslo 2020 канала NDC Conferences
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
![NDC Minnesota - 15-18 Nov 2022 - 4-Day Workshop Event for Software Developers](https://i.ytimg.com/vi/rUP1Hl9jrQg/default.jpg)
![NDC Security Australia - Troy Hunt](https://i.ytimg.com/vi/ZTDZJGMMa_Q/default.jpg)
![Meet Mark Rendle at NDC London 2022](https://i.ytimg.com/vi/gvMd45orRXo/default.jpg)
![Domain-Driven Design & Team Topologies For Product-led Organizations - Nick Tune - NDC Oslo 2022](https://i.ytimg.com/vi/dwtVgOvGcQg/default.jpg)
![NDC London 2019 - Conference for Software Developers](https://i.ytimg.com/vi/1hNs5svtks4/default.jpg)
![Building a Robot Arm with .NET 5, Raspberry Pi, Blazor and Signal R - Peter Gallagher](https://i.ytimg.com/vi/RY7cb-BQO8Q/default.jpg)
![Protect Yourself Against Supply Chain Attacks - Rob Bos - NDC Security 2022](https://i.ytimg.com/vi/00R1JGBQEJg/default.jpg)
![Serverless Patterns Made Simple with Real World Usecases - Sheen Brisals - NDC London 2022](https://i.ytimg.com/vi/mIa5aS7cTig/default.jpg)
![Tomas Jansson - Self service infrastructure using pulumi automation - NDC Oslo 2021](https://i.ytimg.com/vi/xfcOidIqBh8/default.jpg)
![OAuth – the good Parts - Dominick Baier - NDC Oslo 2021](https://i.ytimg.com/vi/y2Psj8ACZyw/default.jpg)
![Forget about OAuth 2.0. Here comes OAuth 2.1 - Philippe De Ryck - NDC Oslo 2022](https://i.ytimg.com/vi/Z9DJzVJD_vg/default.jpg)
![SAFE Stack: The Pit of Success for Functional Web Programming - Isaac Abraham - NDC Oslo 2021](https://i.ytimg.com/vi/Ih5r5qD7bTo/default.jpg)
![Stop Caring about Kubernetes with Azure Container Apps & Dapr - Todd Whitehead - NDC Melbourne 2022](https://i.ytimg.com/vi/vNz7U8r5wjc/default.jpg)
![Building confidence in concurrent code with a model checker - Scott Wlaschin - NDC Oslo 2020](https://i.ytimg.com/vi/tqwcz-Yt9gQ/default.jpg)
![Secure Coding in C and C++ - Volatility Ahead - Robert Seacord - NDC TechTown 2021](https://i.ytimg.com/vi/fVzk4JY2w-M/default.jpg)
![Introduction to Digital Twins with the Azure Digital Twin Platform - Alex Mackey - NDC Melbourne](https://i.ytimg.com/vi/nnuLeQNfruA/default.jpg)
![GraphQL Observability with Elastic and OpenTelemetry - Michael Staib - NDC Porto 2022](https://i.ytimg.com/vi/nCLSfJMihsg/default.jpg)
![NDC Minnesota 2018](https://i.ytimg.com/vi/kDFvYqCNkvQ/default.jpg)
![Getting API Security Right - Philippe De Ryck - NDC Security 2022](https://i.ytimg.com/vi/CSe2FNy7Rko/default.jpg)
![Rethinking Reactive Architectures - David Leitner - NDC Oslo 2021](https://i.ytimg.com/vi/L1Zgfr6bFvA/default.jpg)