Загрузка страницы

LiveCap: Real-time Human Performance Capture from Monocular Video (ACM TOG 2019)

We present the first real-time human performance capture approach that reconstructs dense, space-time coherent deforming geometry of entire humans in general everyday clothing from just a single RGB video.We propose a novel two-stage analysis-by-synthesis optimization whose formulation and implementation are designed for high performance. In the first stage, a skinned template model is jointly fitted to background subtracted input video, 2D and 3D skeleton joint positions found using a deep neural network, and a set of sparse facial landmark detections. In the second stage, dense non-rigid 3D deformations of skin and even loose apparel are captured based on a novel real-time capable algorithm for non-rigid tracking using dense photometric and silhouette constraints. Our novel energy formulation leverages automatically identified material regions on the template to model the differing non-rigid deformation behavior of skin and apparel. The two resulting nonlinear optimization problems per-frame are solved with specially-tailored data-parallel Gauss-Newton solvers. In order to achieve real-time performance of over 25Hz, we design a pipelined parallel architecture using the CPU and two commodity GPUs. Our method is the first real-time monocular approach for full-body performance capture. Our method yields comparable accuracy with off-line performance capture techniques, while being orders of magnitude faster.

M. Habermann, W. Xu, M. Zollhöfer, G. Pons-Moll and C. Theobalt,
LiveCap: Real-time Human Performance Capture from Monocular Video, ACM Transactions on Graphics (to be presented at SIGGRAPH 2019)

Видео LiveCap: Real-time Human Performance Capture from Monocular Video (ACM TOG 2019) канала Christian Theobalt
Показать
Комментарии отсутствуют
Введите заголовок:

Введите адрес ссылки:

Введите адрес видео с YouTube:

Зарегистрируйтесь или войдите с
Информация о видео
6 февраля 2019 г. 18:59:52
00:07:26
Другие видео канала
VolumeDeform: Real-time Volumetric Non-rigid Reconstruction - ECCV 2016VolumeDeform: Real-time Volumetric Non-rigid Reconstruction - ECCV 2016FML: Face Model Learning from Videos (short version) - CVPR 2019FML: Face Model Learning from Videos (short version) - CVPR 2019Corrective 3D reconstruction of lips from monocular video, additional results, SIGGRAPH Asia 2016Corrective 3D reconstruction of lips from monocular video, additional results, SIGGRAPH Asia 2016Real-time Halfway Domain Reconstruction of Motion and Geometry - 3DV 2016Real-time Halfway Domain Reconstruction of Motion and Geometry - 3DV 2016FaceInCar: Real-time Dense Monocular Face Tracking of a Driver, Demo: National IT Summit 2016FaceInCar: Real-time Dense Monocular Face Tracking of a Driver, Demo: National IT Summit 2016Preference and Artifact Analysis of Video Transitions of Places (TAP 2013)Preference and Artifact Analysis of Video Transitions of Places (TAP 2013)HQ3DAvatar: High Quality Controllable 3D Head Avatar, TOG 2024HQ3DAvatar: High Quality Controllable 3D Head Avatar, TOG 2024Deep Relightable Textures: Volumetric Performance Capture with Neural Rendering (SIGGRAPH Asia 2020)Deep Relightable Textures: Volumetric Performance Capture with Neural Rendering (SIGGRAPH Asia 2020)Illumination-invariant Robust Multiview 3D Human Motion Capture - WACV2018Illumination-invariant Robust Multiview 3D Human Motion Capture - WACV2018IMoS Intent Driven Full Body Motion Synthesis for Human Object Interactions. In EUROGRAPHICS, 2023.IMoS Intent Driven Full Body Motion Synthesis for Human Object Interactions. In EUROGRAPHICS, 2023.Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB Input - 3DV 2018Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB Input - 3DV 2018ROAM: Robust and Object-aware Motion Generation using Neural Pose Descriptors, 3DV, 2024ROAM: Robust and Object-aware Motion Generation using Neural Pose Descriptors, 3DV, 2024Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data - CVPR 2020Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data - CVPR 2020Decaf: Monocular Deformation Capture for Face and Hand Interactions. In SIGGRAPH ASIA, 2023.Decaf: Monocular Deformation Capture for Face and Hand Interactions. In SIGGRAPH ASIA, 2023.Performance Capture of High-Speed Motion Using Staggered Multi-view Recording - PG 2012Performance Capture of High-Speed Motion Using Staggered Multi-view Recording - PG 2012PIE: Portrait Image Embedding for Semantic Control (SIGGRAPH Asia 2020)PIE: Portrait Image Embedding for Semantic Control (SIGGRAPH Asia 2020)AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatars. In SIGGRAPH ASIA 2023.AvatarStudio: Text-driven Editing of 3D Dynamic Human Head Avatars. In SIGGRAPH ASIA 2023.Capturing Relightable Human Performances under General Uncontrolled Illumination - EUROGRAPHICS 2013Capturing Relightable Human Performances under General Uncontrolled Illumination - EUROGRAPHICS 2013Sparse localized deformation components - SIGGRAPH Asia 2013Sparse localized deformation components - SIGGRAPH Asia 2013Monocular Real-time Full Body Capture with Inter-part Correlations. In CVPR, 2021Monocular Real-time Full Body Capture with Inter-part Correlations. In CVPR, 2021BundleFusion, Arxiv 2016BundleFusion, Arxiv 2016
Яндекс.Метрика