Allen School Colloquium: Pang Wei Koh (Stanford University)
Tuesday, February 22, 2022, 3:30 pm
Abstract
Machine learning systems are widely deployed today, but they are unreliable. They can fail – and with catastrophic consequences – on subpopulations of the data, such as particular demographic groups, or when deployed in different environments from what they were trained on. In this talk, I will describe our work towards building reliable machine learning systems that are robust to these failures. First, I will show how we can use influence functions to understand the predictions and failures of existing models through the lens of their training data. Second, I will discuss the use of distributionally robust optimization to train models that perform well across all subpopulations. Third, I will describe WILDS – a benchmark of in-the-wild distribution shifts spanning applications such as pathology, conservation, remote sensing, and drug discovery – and show how current state-of-the-art methods, which perform well on synthetic distribution shifts, still fail to be robust on these real-world shifts. Finally, I will describe our work on building more reliable COVID-19 models, using anonymized cellphone mobility data, to inform public health policy; this is a challenging application as the underlying environment is often changing and there is substantial heterogeneity across demographic subpopulations.
Bio
Pang Wei Koh is a PhD student at Stanford, advised by Percy Liang. He studies the theory and practice of building reliable machine learning systems. His research has been published in Nature and Cell, featured in media outlets such as The New York Times and The Washington Post, and recognized by best paper awards at ICML and KDD, a Meta Research PhD fellowship, and the Kennedy Prize for best honors thesis at Stanford. Prior to his PhD, he was the 3rd employee and Director of Partnerships at Coursera.
Видео Allen School Colloquium: Pang Wei Koh (Stanford University) канала Paul G. Allen School
Abstract
Machine learning systems are widely deployed today, but they are unreliable. They can fail – and with catastrophic consequences – on subpopulations of the data, such as particular demographic groups, or when deployed in different environments from what they were trained on. In this talk, I will describe our work towards building reliable machine learning systems that are robust to these failures. First, I will show how we can use influence functions to understand the predictions and failures of existing models through the lens of their training data. Second, I will discuss the use of distributionally robust optimization to train models that perform well across all subpopulations. Third, I will describe WILDS – a benchmark of in-the-wild distribution shifts spanning applications such as pathology, conservation, remote sensing, and drug discovery – and show how current state-of-the-art methods, which perform well on synthetic distribution shifts, still fail to be robust on these real-world shifts. Finally, I will describe our work on building more reliable COVID-19 models, using anonymized cellphone mobility data, to inform public health policy; this is a challenging application as the underlying environment is often changing and there is substantial heterogeneity across demographic subpopulations.
Bio
Pang Wei Koh is a PhD student at Stanford, advised by Percy Liang. He studies the theory and practice of building reliable machine learning systems. His research has been published in Nature and Cell, featured in media outlets such as The New York Times and The Washington Post, and recognized by best paper awards at ICML and KDD, a Meta Research PhD fellowship, and the Kennedy Prize for best honors thesis at Stanford. Prior to his PhD, he was the 3rd employee and Director of Partnerships at Coursera.
Видео Allen School Colloquium: Pang Wei Koh (Stanford University) канала Paul G. Allen School
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
![2019 ADSI Summer Workshop: Algorithmic Foundations of Learning and Control, Mengdi Wang](https://i.ytimg.com/vi/9ulRfhTzqTg/default.jpg)
![Allen School Colloquia: Martin Maas (UC Berkeley)](https://i.ytimg.com/vi/4qtSigNzT4E/default.jpg)
![UW Allen School Colloquium: Arvind Narayanan (Princeton University)](https://i.ytimg.com/vi/WY7bBWD6-Zs/default.jpg)
![Fall 2019 Natural Language Processing: Sam Bowman (NYU)](https://i.ytimg.com/vi/1CoLguOswLo/default.jpg)
![Emeritus Lecture: Oren Etzioni (Oct 2022)](https://i.ytimg.com/vi/Bgq4saV6giQ/default.jpg)
![Software for Embedded Systems, CSE 466, Winter 2008](https://i.ytimg.com/vi/MPh2gTjSq40/default.jpg)
![Allen School Colloquium: Accessibility Research Group at the Allen School](https://i.ytimg.com/vi/iMj9yz24gP8/default.jpg)
![Robots that Talk and Learn (Shiwali Mohan, Palo Alto Research Canter)](https://i.ytimg.com/vi/rJuAC1PUtLs/default.jpg)
![Animation Capstone, CSE460, Winter 2017](https://i.ytimg.com/vi/_dDc0qSZ_bM/default.jpg)
![Allen School Colloquium: ICTD Research Group at the Allen School](https://i.ytimg.com/vi/nOI3nj_Ne_U/default.jpg)
![UW CSE Robotics: Sean Andrist, "Gaze Mechanisms for Situated Interaction with Embodied Agents"](https://i.ytimg.com/vi/GcsmlcjSC0M/default.jpg)
![Allen School Colloquia: Jeff Nivala (University of California - Santa Cruz/Harvard Medical School)](https://i.ytimg.com/vi/DdnQdvC7Rvw/default.jpg)
![Underwater Acoustic Tracking (Ivan Masmitja, Universitat Politècnica de Catalunya)](https://i.ytimg.com/vi/uz0MOzsFcsY/default.jpg)
![UW Allen School Colloquium: Jacob Steinhardt (Stanford University)](https://i.ytimg.com/vi/Pdk3RidEjMY/default.jpg)
![Innovations in Theoretical Computer Science 2020 Session 4](https://i.ytimg.com/vi/kcYXpvsTIaY/default.jpg)
![Allen School Colloquium: Andrea Wei Coladangelo (UC Berkeley)](https://i.ytimg.com/vi/_9ZXOHkMxTc/default.jpg)
![Allen School Colloquia: Fabrication/Access](https://i.ytimg.com/vi/Q9YIajUst6g/default.jpg)
![2019 ADSI Summer Workshop: Algorithmic Foundations of Learning and Control, Pablo Parrilo](https://i.ytimg.com/vi/83R63nfYgJM/default.jpg)
![Fall 2018 Robotics Seminar, Brittany Duncan](https://i.ytimg.com/vi/ycjgjRFFgkA/default.jpg)
![HCI Capstone, CSE 441, Spring 2015](https://i.ytimg.com/vi/xz_rx9hj6o4/default.jpg)
![I Am CSE Alumni Overview](https://i.ytimg.com/vi/WgoDGoHnS40/default.jpg)