Data Stream Processing Concepts and Implementations by Matthias Niehoff
In this talk I will give an overview on various concepts used in data stream processing. Most of them are used for solving problems in the field of time, focussing on processing time compared to event time. The techniques shown include the Dataflow API as it was introduced by Google and the concepts of stream and table duality. But I will also come up with other problems like data lookup and deployment of streaming applications and various strategies on solving these problems.
In the end I will give a brief outline on the implementation status of those strategies in the popular streaming frameworks Apache Spark Streaming, Apache Flink and Kafka Streams.
Matthias Niehoff is an IT consultant at codecentric AG in Germany, where he focuses on big data and streaming applications with Apache Cassandra and Apache Spark as well as other tools in the area of big data. Matthias shares his experience at conferences, meetups, and user groups.
Видео Data Stream Processing Concepts and Implementations by Matthias Niehoff канала Devoxx
In the end I will give a brief outline on the implementation status of those strategies in the popular streaming frameworks Apache Spark Streaming, Apache Flink and Kafka Streams.
Matthias Niehoff is an IT consultant at codecentric AG in Germany, where he focuses on big data and streaming applications with Apache Cassandra and Apache Spark as well as other tools in the area of big data. Matthias shares his experience at conferences, meetups, and user groups.
Видео Data Stream Processing Concepts and Implementations by Matthias Niehoff канала Devoxx
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
Martin Kleppmann — Event Sourcing and Stream Processing at ScaleBuilding Streaming Microservices with Apache Kafka - Tim BerglundStreaming Concepts & Introduction to Flink Series - What is Stream Processing & Apache FlinkApache Kafka and KSQL in Action : Let’s Build a Streaming Data Pipeline! by Robin MoffattBatch Processing vs Stream Processing | System Design Primer | Tech PrimersDeep Learning For Real Time Streaming Data With Kafka And Tensorflow | YongTang - ODSC East 2019High Performance Data Streaming with Amazon Kinesis: Best Practices and Common Pitfalls[VDT19] Where is my cache? Architectural patterns for caching microservices by exampleStream Processing – Concepts and Frameworks (Guido Schmutz, Switzerland)Everything You Need to Know About Big Data: From Architectural Principles to Best PracticesBuilding a hybrid data pipeline using Kafka and ConfluentKafka Tutorial - Core ConceptsAWS re:Invent 2017: Netflix Keystone SPaaS: Real-time Stream Processing as a Service (ABD320)GOTO 2019 • Introduction to Stateful Stream Processing with Apache Flink • Robert MetzgerStream Processing Design Patterns | Capital OneAmazon Kinesis Data Streams: Why Streaming Data?A Modern Data Pipeline in Action (Cloud Next '18)Spark Streaming Example with PySpark ❌ BEST Apache SPARK Structured STREAMING TUTORIAL with PySparkGCP for Apache Kafka Users: Stream Ingestion and Processing (Cloud Next '19)