2021-12-06 - Ryan Chard - funcX: Federated FaaS for Scientific Computing
NERSC Data Seminars Series: https://github.com/NERSC/data-seminars
Title:
funcX: Federated FaaS for Scientific Computing
Speaker:
Ryan Chard
Data Science and Learning Division
Argonne National Laboratory
Abstract:
Exploding data volumes and velocities, new computational methods and platforms, and ubiquitous connectivity demand new approaches to computation in the sciences. These new approaches must enable computation to be mobile, so that, for example, it can occur near data, be triggered by events (e.g., arrival of new data), be offloaded to specialized accelerators, or run remotely where resources are available. They also require new design approaches in which monolithic applications can be decomposed into smaller components, that may in turn be executed separately and on the most suitable resources. To address these needs we present funcX—a distributed function as a service (FaaS) platform that enables flexible, scalable, and high-performance remote function execution. funcX's endpoint software can transform existing clusters and supercomputers into function serving systems, while funcX's cloud-hosted service provides transparent, secure, and reliable function execution across a federated ecosystem of endpoints. We demonstrate the use of funcX with several scientific case studies and show how it integrates into the wider Globus ecosystem to enable secure, fire-and-forget scientific computation.
Bio:
Ryan Chard joined Argonne in 2016 as a Maria Goeppert Mayer Fellow and then as an Assistant Computer Scientist in the Data Science and Learning Division. He now works with Argonne, UChicago, and Globus to develop cyberinfrastructure to enable scientific research. In particular, he works on the Globus Flows platform to create reliable data analysis pipelines and the funcX service to enable function serving for HPC. He has a Ph.D. in computer science and an M.Sc. from Victoria University of Wellington, New Zealand.
Host of Seminar:
Jonathan Skone, Advanced Technologies
Bjoern Enders, Data Science Engagement Group
National Energy Research Scientific Computing Center (NERSC)
Lawrence Berkeley National Laboratory
Видео 2021-12-06 - Ryan Chard - funcX: Federated FaaS for Scientific Computing канала NERSC
Title:
funcX: Federated FaaS for Scientific Computing
Speaker:
Ryan Chard
Data Science and Learning Division
Argonne National Laboratory
Abstract:
Exploding data volumes and velocities, new computational methods and platforms, and ubiquitous connectivity demand new approaches to computation in the sciences. These new approaches must enable computation to be mobile, so that, for example, it can occur near data, be triggered by events (e.g., arrival of new data), be offloaded to specialized accelerators, or run remotely where resources are available. They also require new design approaches in which monolithic applications can be decomposed into smaller components, that may in turn be executed separately and on the most suitable resources. To address these needs we present funcX—a distributed function as a service (FaaS) platform that enables flexible, scalable, and high-performance remote function execution. funcX's endpoint software can transform existing clusters and supercomputers into function serving systems, while funcX's cloud-hosted service provides transparent, secure, and reliable function execution across a federated ecosystem of endpoints. We demonstrate the use of funcX with several scientific case studies and show how it integrates into the wider Globus ecosystem to enable secure, fire-and-forget scientific computation.
Bio:
Ryan Chard joined Argonne in 2016 as a Maria Goeppert Mayer Fellow and then as an Assistant Computer Scientist in the Data Science and Learning Division. He now works with Argonne, UChicago, and Globus to develop cyberinfrastructure to enable scientific research. In particular, he works on the Globus Flows platform to create reliable data analysis pipelines and the funcX service to enable function serving for HPC. He has a Ph.D. in computer science and an M.Sc. from Victoria University of Wellington, New Zealand.
Host of Seminar:
Jonathan Skone, Advanced Technologies
Bjoern Enders, Data Science Engagement Group
National Energy Research Scientific Computing Center (NERSC)
Lawrence Berkeley National Laboratory
Видео 2021-12-06 - Ryan Chard - funcX: Federated FaaS for Scientific Computing канала NERSC
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
Best Practices for HPC Software Developers: #7 "Basic Performance Analysis & Optimization"6 Case Study 1: XGC1 (NERSC Cori KNL Training 6/2017)21 - Scaling Neural Networks Training - Thorsten Kurth2022-06-14 - Jihan Kim - Artificial Design of Porous Materials2021-08-24 - Kjiersten Fagnan - JGI Computing - the future is looking cloudy13 Jupyter at NERSCJupyter demo videoCheckpoint/Restart MPI Applications with MANA on CoriCurrent State of LLVM CompilerJuliaExploring the Dark Universe with PerlmutterApplication of GPUs in Proteomics and ConnectomicsMarch 2022 NUG Meeting: Remote development in NERSC Shifter containers with Visual Studio CodeDocker Containers & Dark Matter: Overview Of the Spin Container Platform with Highlights from LZ03 Codee parallelization capabilities2021-03-09 - Shane Snyder - Darshan: Enabling IO Understanding in an Evolving HPC Landscape8 Homework ATMUX exercise demoInteractive Distributed Computing with Jupyter and FriendsCurrent State of CUDA Compilers and SDKIntro to GPU: 04 NVidia Software Stack, Part 2