Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data - CVPR 2020
Abstract
We present a novel method for monocular hand shape and pose estimation at unprecedented runtime performance of 100fps and at state-of-the-art accuracy. This is enabled by a new learning based architecture designed such that it can make use of all the sources of available hand training data: image data with either 2D or 3D annotations, as well as stand-alone 3D animations without corresponding image data. It features a 3D hand joint detection module and an inverse kinematics module which regresses not only 3D joint positions but also maps them to joint rotations in a single feed-forward pass. This output makes the method more directly usable for applications in computer vision and graphics compared to only regressing 3D joint positions. We demonstrate that our architectural design leads to a significant quantitative and qualitative improvement over the state of the art on several challenging benchmarks.
Y. Zhou, M. Habermann, W. Xu, V. Golyanik, C. Theobalt and F. Xu
Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data, Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2020
Видео Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data - CVPR 2020 канала Christian Theobalt
We present a novel method for monocular hand shape and pose estimation at unprecedented runtime performance of 100fps and at state-of-the-art accuracy. This is enabled by a new learning based architecture designed such that it can make use of all the sources of available hand training data: image data with either 2D or 3D annotations, as well as stand-alone 3D animations without corresponding image data. It features a 3D hand joint detection module and an inverse kinematics module which regresses not only 3D joint positions but also maps them to joint rotations in a single feed-forward pass. This output makes the method more directly usable for applications in computer vision and graphics compared to only regressing 3D joint positions. We demonstrate that our architectural design leads to a significant quantitative and qualitative improvement over the state of the art on several challenging benchmarks.
Y. Zhou, M. Habermann, W. Xu, V. Golyanik, C. Theobalt and F. Xu
Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data, Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2020
Видео Monocular Real-time Hand Shape and Motion Capture using Multi-modal Data - CVPR 2020 канала Christian Theobalt
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
![MoreFusion: Multi-object Reasoning for 6D Pose Estimation from Volumetric Fusion](https://i.ytimg.com/vi/6oLUhuZL4ko/default.jpg)
![XNect: Real-time Multi-person 3D Motion Capture With a Single RGB Camera (SIGGRAPH 2020)](https://i.ytimg.com/vi/tdxMo11KJGk/default.jpg)
![Real-time Pose and Shape Reconstruction of Two Hands With a Single Depth Camera SIGGRAPH'19](https://i.ytimg.com/vi/mfCXqvP9cAQ/default.jpg)
![A.I. Experiments: Visualizing High-Dimensional Space](https://i.ytimg.com/vi/wvsE8jm1GzE/default.jpg)
![RGB2Hands: Real-Time Tracking of 3D Hand Interactions from Monocular RGB Video (SIGGRAPH Asia 2020)](https://i.ytimg.com/vi/7qt6ZSo4-JY/default.jpg)
![Free face tracking update: New UI, Eyebrow tracking, new Blender and Unreal Files...and more!](https://i.ytimg.com/vi/I4qkhuQk3_c/default.jpg)
![MarI/O - Machine Learning for Video Games](https://i.ytimg.com/vi/qv6UVOQ0F44/default.jpg)
![DeepCap: Monocular Human Performance Capture Using Weak Supervision (CVPR 2020) - Oral](https://i.ytimg.com/vi/C4eDrvJ9aBs/default.jpg)
![FingerTrak: Continuous 3D Hand Pose Tracking by Deep Learning Hand Silhouettes Captured by ...](https://i.ytimg.com/vi/XC16TCRCUss/default.jpg)
![3D markerless tracking: Running // CONTEMPLAS MULTI CAMERA SYSTEM®](https://i.ytimg.com/vi/8RZQIAyZhaU/default.jpg)
![Theia3D - Markerless Motion Capture Redefined](https://i.ytimg.com/vi/CGfkZLz6dPE/default.jpg)
![CVPR 2020 Elevator Pitch of the Autonomous Vision Group](https://i.ytimg.com/vi/zz1ok-viqgs/default.jpg)
![The Hand of Animation - Hjalti Hjalmarsson](https://i.ytimg.com/vi/3qj0ZAX61Ho/default.jpg)
![Real-time Deep Dynamic Characters (SIGGRAPH 2021)](https://i.ytimg.com/vi/txw5FIU8uYk/default.jpg)
![Best AI Motion Capture 2021 - OpenPose vs DeepMotion](https://i.ytimg.com/vi/T1vvokFnsbU/default.jpg)
![Theia Markerless: Welcome to the Future of Motion Capture](https://i.ytimg.com/vi/S_dUqepsJTY/default.jpg)
![3D hand pose estimation from single RGB image - Olga Chernytska](https://i.ytimg.com/vi/94UBrCmn6MY/default.jpg)
![[Talk] Analyzing CNN Artifacts in Discriminative and Generative Models (02/20)](https://i.ytimg.com/vi/aNDwHRxWTa0/default.jpg)
![Wukong Modeling - Blender 2.9](https://i.ytimg.com/vi/mcevQuXJurM/default.jpg)
![Automatic Face Reenactment - CVPR 2014](https://i.ytimg.com/vi/rGiFi4Kqk3s/default.jpg)