TUM AI Lecture Series - Understanding and Extending Neural Radiance Fields (Jonathan T. Barron)

Abstract: Neural Radiance Fields (Mildenhall, Srinivasan, Tancik, et al., ECCV 2020) are an effective and simple technique for synthesizing photorealistic novel views of complex scenes by optimizing an underlying continuous volumetric radiance field, parameterized by a (non-convolutional) neural network. I will discuss and review NeRF and then introduce two works that closely relate to it: First, I will explain why NeRF (and other CPPN-like architectures that map from low-dimensional coordinates to intensities) depend critically on the use of a trigonometric "positional encoding", aided by insights provided by the neural tangent kernel literature. Second, I will show how NeRF can be extended to incorporate explicit reasoning about occluders and appearance variation, and can thereby enable photorealistic view synthesis and photometric manipulation using only unstructured image collections.

Bio: Jon Barron is a staff research scientist at Google, where he works on computer vision and machine learning. He received a Ph.D. in Computer Science from the University of California, Berkeley in 2013, where he was advised by Jitendra Malik, and he received an Honours BSc in Computer Science from the University of Toronto in 2007. He received a National Science Foundation Graduate Research Fellowship in 2009, the C.V. Ramamoorthy Distinguished Research Award in 2013, the PAMI Young Researcher Award in 2020, and the ECCV Best Paper Honorable Mention in both 2016 and 2020.

TUM AI Lecture Series previous lectures:
- Towards Graph-Based Spatial AI (Andrew Davison): https://youtu.be/_npGEB3kkVc
- Reconstructing the Plenoptic Function (Noah Snavely): https://youtu.be/GNUpZAeBnZc
- Neural Implicit Representations for 3D Vision (Andreas Geiger): https://youtu.be/F9mRv4v80w0

TUM AI Lecture Series upcoming lectures:
- The Moon Camera (Bill Freeman): https://youtu.be/Ytkkl917paM

Видео TUM AI Lecture Series - Understanding and Extending Neural Radiance Fields (Jonathan T. Barron) канала cvprtum