The Design of Systems for Real-time Prediction Serving | UC Berkeley
Get the slides: https://www.datacouncil.ai/talks/the-design-of-systems-for-real-time-prediction-serving?utm_source=youtube&utm_medium=social&utm_campaign=%20-%20DEC-SF-18%20Slides%20Download
ABOUT THE TALK:
Machine learning is being deployed in a growing number of applications which demand real-time, accurate, and robust predictions under heavy serving loads. However, most machine learning frameworks and systems only address model training and not deployment.
Clipper is an open-source, general-purpose model-serving system that addresses these challenges. Interposing between applications that consume predictions and the machine-learning models that produce predictions, Clipper simplifies the model deployment process by adopting a modular serving architecture and isolating models in their own containers, allowing them to be evaluated using the same runtime environment as that used during training.
Clipper's modular architecture provides simple mechanisms for scaling out models to meet increased throughput demands and performing fine-grained physical resource allocation for each model. Further, by abstracting models behind a uniform serving interface, Clipper allows developers to compose many machine-learning models within a single application to support increasingly common techniques such as ensemble methods, multi-armed bandit algorithms, and prediction cascades.
In this talk Joey will provide an overview of the Clipper serving system and discuss their experience transforming a research prototype into an active, open source system. He will then discuss some recent work on end-to-end cost-aware resource allocation and scheduling for multi-model applications.
ABOUT THE SPEAKER:
Joseph Gonzalez is a professor in the Electrical Engineering and Computer Science Department at UC Berkeley and co-director of the UC Berkeley RISELab. Joseph studies the design of algorithms and systems for scalable and secure machine learning (ML).
His research has applications in autonomous vehicles, recommendation systems, database management systems, robotics, personalized medicine, and intelligent assistants. Joseph also teaches the advanced data science class at UC Berkeley to over 600 students a semester and is helping to develop the new data science major.
Joseph is on the technical advisory board for Deepscale.ai. Before joining UC Berkeley, Joseph co-founded Turi Inc. (formerly GraphLab) to develop AI tools for data scientists and later sold Turi to Apple. He holds a PhD in Machine Learning from Carnegie Mellon University and is a member of the Apache Spark PMC.
ABOUT DATA COUNCIL:
Data Council (https://www.datacouncil.ai/) is a community and conference series that provides data professionals with the learning and networking opportunities they need to grow their careers. Make sure to subscribe to our channel for more videos, including DC_THURS, our series of live online interviews with leading data professionals from top open source projects and startups.
FOLLOW DATA COUNCIL:
Twitter: https://twitter.com/DataCouncilAI
LinkedIn: https://www.linkedin.com/company/datacouncil-ai
Facebook: https://www.facebook.com/datacouncilai
Eventbrite: https://www.eventbrite.com/o/data-council-30357384520
Видео The Design of Systems for Real-time Prediction Serving | UC Berkeley канала Data Council
ABOUT THE TALK:
Machine learning is being deployed in a growing number of applications which demand real-time, accurate, and robust predictions under heavy serving loads. However, most machine learning frameworks and systems only address model training and not deployment.
Clipper is an open-source, general-purpose model-serving system that addresses these challenges. Interposing between applications that consume predictions and the machine-learning models that produce predictions, Clipper simplifies the model deployment process by adopting a modular serving architecture and isolating models in their own containers, allowing them to be evaluated using the same runtime environment as that used during training.
Clipper's modular architecture provides simple mechanisms for scaling out models to meet increased throughput demands and performing fine-grained physical resource allocation for each model. Further, by abstracting models behind a uniform serving interface, Clipper allows developers to compose many machine-learning models within a single application to support increasingly common techniques such as ensemble methods, multi-armed bandit algorithms, and prediction cascades.
In this talk Joey will provide an overview of the Clipper serving system and discuss their experience transforming a research prototype into an active, open source system. He will then discuss some recent work on end-to-end cost-aware resource allocation and scheduling for multi-model applications.
ABOUT THE SPEAKER:
Joseph Gonzalez is a professor in the Electrical Engineering and Computer Science Department at UC Berkeley and co-director of the UC Berkeley RISELab. Joseph studies the design of algorithms and systems for scalable and secure machine learning (ML).
His research has applications in autonomous vehicles, recommendation systems, database management systems, robotics, personalized medicine, and intelligent assistants. Joseph also teaches the advanced data science class at UC Berkeley to over 600 students a semester and is helping to develop the new data science major.
Joseph is on the technical advisory board for Deepscale.ai. Before joining UC Berkeley, Joseph co-founded Turi Inc. (formerly GraphLab) to develop AI tools for data scientists and later sold Turi to Apple. He holds a PhD in Machine Learning from Carnegie Mellon University and is a member of the Apache Spark PMC.
ABOUT DATA COUNCIL:
Data Council (https://www.datacouncil.ai/) is a community and conference series that provides data professionals with the learning and networking opportunities they need to grow their careers. Make sure to subscribe to our channel for more videos, including DC_THURS, our series of live online interviews with leading data professionals from top open source projects and startups.
FOLLOW DATA COUNCIL:
Twitter: https://twitter.com/DataCouncilAI
LinkedIn: https://www.linkedin.com/company/datacouncil-ai
Facebook: https://www.facebook.com/datacouncilai
Eventbrite: https://www.eventbrite.com/o/data-council-30357384520
Видео The Design of Systems for Real-time Prediction Serving | UC Berkeley канала Data Council
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
Deep Recommender Systems at Facebook feat. Carole-Jean Wu | Stanford MLSys Seminar Episode 24Real-Time Data Pipelines Made Easy with Structured Streaming in Apache Spark | DatabricksSystem Design: Content Delivery Networks (Simplified)The 8 Best Examples Of Real-Time Data Analytics5 Design Patterns Every Engineer Should KnowSystem Design Interview - Notification ServiceThe incredible inventions of intuitive AI | Maurice ContiHow to Design and Build a Recommendation System Pipeline in Python (Jill Cates)System Design Interview - Distributed CacheThe Microsoft Data Science InterviewImportance of Data Annotation Services | Data Labeling Services | Learning Spiral AIData Lineage (Data Architecture | Data Governance)Large-Scale Low-Latency Storage for the Social Network - Data@ScaleEase ML deployments with TensorFlow Serving - Cambridge ML Summit ‘19Machine Learning design: Search engine for Q&AWhat is a Data Architecture? Modern Data Architectures ExplainedWhat is and why data architectureAccelerating Development Velocity of Production ML Systems with DockerDistributed Systems in One Lesson by Tim Berglund