Stream Processing – Concepts and Frameworks (Guido Schmutz, Switzerland)
https://jeeconf.com/program/stream-processing-concepts-and-frameworks/
More and more data sources today provide a constant stream of data, from IoT devices to Social Media streams. It is one thing to collect these events in the velocity they arrive, without losing any single message. An Event Hub and a data flow engine can help here. It’s another thing to do some (complex) analytics on the data. There is always the option to first store in a data sink of choice and later analyze it. Storing even a high-volume event stream is feasible and not a challenge anymore. But this adds to the end-to-end latency and it takes minutes if not hours to present results. If you need to react fast, you simply can’t afford to first store the data. You need to do process it directly on the data stream. This is called Stream Processing or Stream Analytics. In this talk I will present the important concepts, a Stream Processing solution should support and then dive into some of the most popular frameworks available on the market and how they compare.
Видео Stream Processing – Concepts and Frameworks (Guido Schmutz, Switzerland) канала jeeconf
More and more data sources today provide a constant stream of data, from IoT devices to Social Media streams. It is one thing to collect these events in the velocity they arrive, without losing any single message. An Event Hub and a data flow engine can help here. It’s another thing to do some (complex) analytics on the data. There is always the option to first store in a data sink of choice and later analyze it. Storing even a high-volume event stream is feasible and not a challenge anymore. But this adds to the end-to-end latency and it takes minutes if not hours to present results. If you need to react fast, you simply can’t afford to first store the data. You need to do process it directly on the data stream. This is called Stream Processing or Stream Analytics. In this talk I will present the important concepts, a Stream Processing solution should support and then dive into some of the most popular frameworks available on the market and how they compare.
Видео Stream Processing – Concepts and Frameworks (Guido Schmutz, Switzerland) канала jeeconf
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
Building event-driven (Micro)Services with Apache Kafka by Guido SchmutzBatch Processing vs Stream Processing | System Design Primer | Tech PrimersIntroduction to Stateful Stream Processing with Apache Flink • Robert Metzger • GOTO 2019The Rise Of Open-Source SoftwareApache Arrow: In Theory, In Practice // Apache Arrow Meetup SFLessons learned form Kafka in production (Tim Berglund, Confluent)Building data lakes on Google CloudFuture of Data EngineeringData Stream Processing Concepts and Implementations by Matthias NiehoffWasteful waste or why everything is usually so slow in development (Mikalai Alimenkou, Ukraine) [RU]High Performance Data Streaming with Amazon Kinesis: Best Practices and Common PitfallsMartin Kleppmann — Event Sourcing and Stream Processing at ScaleETL Is Dead, Long Live Streams: real-time streams w/ Apache Kafka"Building a Distributed Task Scheduler With Akka, Kafka, and Cassandra" by David van GeestHow Science is Taking the Luck out of Gambling - with Adam KucharskiNetflix System Design | Media Streaming Platform | System Design InterviewThe Many Meanings of Event-Driven Architecture • Martin Fowler • GOTO 2017Building Streaming Microservices with Apache Kafka - Tim BerglundFour Distributed Systems Architectural Patterns by Tim BerglundStreaming Concepts & Introduction to Flink Series - What is Stream Processing & Apache Flink