Allen School Distinguished Lecture: David Patterson (UC Berkeley/Google)
Lecture Title: Domain Specific Architectures for Deep Neural Networks: Three Generations of Tensor Processing Units (TPUs)
The recent success of deep neural networks (DNN) has inspired a resurgence in domain specific architectures (DSAs) to run them, partially as a result of the deceleration of microprocessor performance improvement due to the ending of Moore's Law.
DNNs have two phases: training, which constructs accurate models, and inference, which serves those models. Google's first generation Tensor Processing Unit (TPUv1) offered 50X improvement in performance per watt over conventional architectures for inference. We naturally asked whether a successor could do the same for training.
This talk reviews TPUv1 and explores how Google built the first production DSA supercomputer for the much harder problem of training, which was deployed in 2017.
Google's TPUv2/TPUv3 supercomputers with up to 1024 chips train production DNNs at close to perfect linear speedup with 10X-40X higher floating point operations per Watt than general-purpose supercomputers running the high-performance computing benchmark Linpack.
Bio
David Patterson is a Berkeley CS professor emeritus, a Google distinguished engineer, and the RISC-V Foundation Vice-Chair. He received his BA, MS, and PhD degrees from UCLA. His Reduced Instruction Set Computer (RISC), Redundant Array of Inexpensive Disks (RAID), and Network of Workstation projects helped lead to multi-billion-dollar industries. This work led to 40 awards for research, teaching, and service plus many papers and seven books. The best known book is Computer Architecture: A Quantitative Approach and the newest is The RISC-V Reader: An Open Architecture Atlas. In 2017 he and John Hennessy shared the ACM A.M. Turing Award.
More information available at www.cs.washington.edu.
Originally recorded on October 31, 2019. This video is closed captioned.
Видео Allen School Distinguished Lecture: David Patterson (UC Berkeley/Google) канала Paul G. Allen School
The recent success of deep neural networks (DNN) has inspired a resurgence in domain specific architectures (DSAs) to run them, partially as a result of the deceleration of microprocessor performance improvement due to the ending of Moore's Law.
DNNs have two phases: training, which constructs accurate models, and inference, which serves those models. Google's first generation Tensor Processing Unit (TPUv1) offered 50X improvement in performance per watt over conventional architectures for inference. We naturally asked whether a successor could do the same for training.
This talk reviews TPUv1 and explores how Google built the first production DSA supercomputer for the much harder problem of training, which was deployed in 2017.
Google's TPUv2/TPUv3 supercomputers with up to 1024 chips train production DNNs at close to perfect linear speedup with 10X-40X higher floating point operations per Watt than general-purpose supercomputers running the high-performance computing benchmark Linpack.
Bio
David Patterson is a Berkeley CS professor emeritus, a Google distinguished engineer, and the RISC-V Foundation Vice-Chair. He received his BA, MS, and PhD degrees from UCLA. His Reduced Instruction Set Computer (RISC), Redundant Array of Inexpensive Disks (RAID), and Network of Workstation projects helped lead to multi-billion-dollar industries. This work led to 40 awards for research, teaching, and service plus many papers and seven books. The best known book is Computer Architecture: A Quantitative Approach and the newest is The RISC-V Reader: An Open Architecture Atlas. In 2017 he and John Hennessy shared the ACM A.M. Turing Award.
More information available at www.cs.washington.edu.
Originally recorded on October 31, 2019. This video is closed captioned.
Видео Allen School Distinguished Lecture: David Patterson (UC Berkeley/Google) канала Paul G. Allen School
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
Allen School Distinguished Lecture: Jeff Dean (Google AI)Dave Patterson Evaluation of the Tensor Processing UnitNew Golden Age for Computer Architectures | Dave Patterson (UC Berkeley)ACM A.M. Turing Award 2017: David Patterson and John Hennessy"A New Golden Age for Computer Architecture" with Dave PattersonNobel Lecture: Jennifer Doudna, Nobel Prize in Chemistry 2020The Darwin Day Lecture 2019, with Richard DawkinsWill Computers Ever Think Like Human Beings? - with Vint CerfDiving into the TPU v2 and v3Understanding Linguistics | Noam Chomsky | Talks at GoogleHow to Have a Bad Career | David Patterson | Talks at GoogleChemical Curiosities: Surprising Science and Dramatic Demonstrations - with Chris BishopConstitution 101 | Lecture 1UC Berkeley CS10 FA10 Lecture 16, Human-Computer Interaction (HCI) with Björn Hartmann (1080p HD)Focus: the Hidden Driver of Excellence | Daniel Goleman | Talks at GoogleThe Exodus from Egypt, a Lecture with Dr. James HoffmeierWho Receives More Government Help, Stanford or UC Berkeley? | Robert ReichTesla Dojo, TPU, NVIDIA & hardware optimized for machine learning | George Hotz and Lex FridmanCloud TPU Pods: AI Supercomputing for Large Machine Learning Problems (Google I/O'19)