Naiad: a timely dataflow system
Naiad is a distributed system for executing data parallel, cyclic dataflow programs. It offers the high throughput of batch processors, the low latency of stream processors, and the ability to perform iterative and incremental computations. Although existing systems offer some of these features, applications that require all three have relied on multiple platforms, at the expense of efficiency, maintainability, and simplicity. Naiad resolves the complexities of combining these features in one framework.
A new computational model, timely dataflow, underlies Naiad and captures opportunities for parallelism across a wide class of algorithms. This model enriches dataflow computation with timestamps that represent logical points in the computation and provide the basis for an efficient, lightweight coordination mechanism.
We show that many powerful high-level programming models can be built on Naiad's low-level primitives, enabling such diverse tasks as streaming data analysis, iterative machine learning, and interactive graph mining. Naiad outperforms specialized systems in their target application domains, and its unique features enable the development of new high-performance applications.
In the ACM Digital Library: http://dl.acm.org/citation.cfm?id=2522738
Видео Naiad: a timely dataflow system канала Association for Computing Machinery (ACM)
A new computational model, timely dataflow, underlies Naiad and captures opportunities for parallelism across a wide class of algorithms. This model enriches dataflow computation with timestamps that represent logical points in the computation and provide the basis for an efficient, lightweight coordination mechanism.
We show that many powerful high-level programming models can be built on Naiad's low-level primitives, enabling such diverse tasks as streaming data analysis, iterative machine learning, and interactive graph mining. Naiad outperforms specialized systems in their target application domains, and its unique features enable the development of new high-performance applications.
In the ACM Digital Library: http://dl.acm.org/citation.cfm?id=2522738
Видео Naiad: a timely dataflow system канала Association for Computing Machinery (ACM)
Показать
Комментарии отсутствуют
Информация о видео
27 февраля 2014 г. 23:25:27
00:27:37
Другие видео канала
Naiad: A System for Incremental, Iterative and Interactive Parallel ComputationTimely dataflow in three easy steps!Dataflow: A Unified Model for Batch and Streaming Data ProcessingX-Stream: edge-centric graph processing using streaming partitionsData Flow Programming BasicsTop 5 Mistakes When Writing Spark ApplicationsSSIS Control Flow vs. Data Flow0.4: What are some programming languages? - Processing Tutorial“Computer Science as a Major Body of Accumulated Knowledge,” at ACM Turing 50 CelebrationBDA - Incremental, Iterative, and Interactive Computation using Differential DataflowIntroducing Project Naiad and Differential DataflowApache Spark vs. Apache FlinkElements of Dataflow and Reactive Programming SystemsSpark Architecture - Part 1Using Graph Partitioning in Distributed Systems DesignIt's About Time: An Introduction to Timely Dataflow | Clockworks"Data Driven UIs, Incrementally" by Yaron MinskyVisual Programming in Data Flow Environments, lecture by Craig Upson#49 Dataflow Analysis