Total Relighting SIGGRAPH Talk (Full Length)
SIGGRAPH 2021 Technical Paper:
Total Relighting: Learning to Relight Portraits for Background Replacement - Rohit Pandey*, Sergio Orts-Escolano*, Chloe LeGendre*, Christian Haene, Sofien Bouaziz, Christoph Rhemann, Paul Debevec, and Sean Fanello
Project Page:
https://augmentedperception.github.io/total_relighting/
We propose a novel system for portrait relighting and background replacement, which maintains high-frequency boundary details and accurately synthesizes the subject's appearance as lit by novel illumination, thereby producing realistic composite images for any desired scene. Our technique includes foreground estimation via alpha matting, relighting, and compositing. We demonstrate that each of these stages can be tackled in a sequential pipeline without the use of priors (e.g. known background or known illumination) and with no specialized acquisition techniques, using only a single RGB portrait image and a novel, target HDR lighting environment as inputs. We train our model using relit portraits of subjects captured in a light stage computational illumination system, which records multiple lighting conditions, high quality geometry, and accurate alpha mattes. To perform realistic relighting for compositing, we introduce a novel per-pixel lighting representation in a deep learning framework, which explicitly models the diffuse and the specular components of appearance, producing relit portraits with convincingly rendered non-Lambertian effects like specular highlights. Multiple experiments and comparisons show the effectiveness of the proposed approach when applied to in-the-wild images.
Видео Total Relighting SIGGRAPH Talk (Full Length) канала Chloe LeGendre
Total Relighting: Learning to Relight Portraits for Background Replacement - Rohit Pandey*, Sergio Orts-Escolano*, Chloe LeGendre*, Christian Haene, Sofien Bouaziz, Christoph Rhemann, Paul Debevec, and Sean Fanello
Project Page:
https://augmentedperception.github.io/total_relighting/
We propose a novel system for portrait relighting and background replacement, which maintains high-frequency boundary details and accurately synthesizes the subject's appearance as lit by novel illumination, thereby producing realistic composite images for any desired scene. Our technique includes foreground estimation via alpha matting, relighting, and compositing. We demonstrate that each of these stages can be tackled in a sequential pipeline without the use of priors (e.g. known background or known illumination) and with no specialized acquisition techniques, using only a single RGB portrait image and a novel, target HDR lighting environment as inputs. We train our model using relit portraits of subjects captured in a light stage computational illumination system, which records multiple lighting conditions, high quality geometry, and accurate alpha mattes. To perform realistic relighting for compositing, we introduce a novel per-pixel lighting representation in a deep learning framework, which explicitly models the diffuse and the specular components of appearance, producing relit portraits with convincingly rendered non-Lambertian effects like specular highlights. Multiple experiments and comparisons show the effectiveness of the proposed approach when applied to in-the-wild images.
Видео Total Relighting SIGGRAPH Talk (Full Length) канала Chloe LeGendre
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
"Multispectral Lighting Reproduction for Virtual Production," Chloe LeGendre, SIGGRAPH 2022 CourseHDR Lighting Dilation for Dynamic Range Reduction on Virtual Production StagesLearning Illumination from Diverse Portraits: SIGGRAPH Asia 2020 Technical CommunicationsLearning Illumination from Diverse Portraits - SIGGRAPH Asia 2020 Technical CommunicationsChloe LeGendre's SIGGRAPH 2019 Thesis Fast Forward submission videoEfficient Multispectral Facial Capture with Monochrome CamerasTotal Relighting: Fast Forward (SIGGRAPH 2021)Learning to Estimate Illumination (CVPR 2019)Improved Chromakey of Hair Strands via Orientation Filter ConvolutionModeling Vellus Facial Hair from Asperity Scattering Silhouettes