3D Face Reconstruction with Dense Landmarks
Landmarks often play a key role in face analysis, but many aspects of identity or expression cannot be represented by sparse landmarks alone. Thus, in order to reconstruct faces more accurately, landmarks are often combined with additional signals like depth images or techniques like differentiable rendering.
Can we keep things simple by just using more landmarks?
In answer, we present the first method that accurately predicts ten times as many landmarks as usual, covering the whole head, including the eyes and teeth. This is accomplished using synthetic training data, which guarantees perfect landmark annotations. By fitting a morphable model to these dense landmarks, we achieve state-of-the-art results for monocular 3D face reconstruction in the wild. We show that dense landmarks are an ideal signal for integrating face shape information across frames by demonstrating accurate and expressive facial performance capture in both monocular and multi-view scenarios. This approach is also highly efficient: we can predict dense landmarks and fit our 3D face model at over 150FPS on a single CPU thread.
Learn more about this work: https://www.microsoft.com/en-us/research/publication/3d-face-reconstruction-with-dense-landmarks/
Microsoft Research's Mixed Reality & AI Lab – Cambridge: https://www.microsoft.com/en-us/research/lab/mixed-reality-ai-lab-cambridge/
Microsoft Research's Mixed Reality & AI Lab – Zurich: https://www.microsoft.com/en-us/research/lab/mixed-reality-ai-zurich/
Видео 3D Face Reconstruction with Dense Landmarks канала Microsoft Research
Can we keep things simple by just using more landmarks?
In answer, we present the first method that accurately predicts ten times as many landmarks as usual, covering the whole head, including the eyes and teeth. This is accomplished using synthetic training data, which guarantees perfect landmark annotations. By fitting a morphable model to these dense landmarks, we achieve state-of-the-art results for monocular 3D face reconstruction in the wild. We show that dense landmarks are an ideal signal for integrating face shape information across frames by demonstrating accurate and expressive facial performance capture in both monocular and multi-view scenarios. This approach is also highly efficient: we can predict dense landmarks and fit our 3D face model at over 150FPS on a single CPU thread.
Learn more about this work: https://www.microsoft.com/en-us/research/publication/3d-face-reconstruction-with-dense-landmarks/
Microsoft Research's Mixed Reality & AI Lab – Cambridge: https://www.microsoft.com/en-us/research/lab/mixed-reality-ai-lab-cambridge/
Microsoft Research's Mixed Reality & AI Lab – Zurich: https://www.microsoft.com/en-us/research/lab/mixed-reality-ai-zurich/
Видео 3D Face Reconstruction with Dense Landmarks канала Microsoft Research
Показать
Комментарии отсутствуют
Информация о видео
Другие видео канала
The Intern Experience at Microsoft Research CambridgeBattling Tuberculosis Using Microsoft TechnologyFast and Flexible Multi-Task Classification Using Conditional Neural Adaptive ProcessesData Visualization Reaches New Heights with LayerscapeFaculty Summit 2018 IntroductionMicrosoft Pix - Take Better Photos of People, AutomaticallyImprovements on Higher Order Ambisonics ReproductionCambridge lab overview with Chris BishopGet free cloud computing time and storage on Microsoft AzureUnlocking Real world solutions with AI – Chris BishopHow interns at our New England Lab impact research at MicrosoftAI Institute "Geometry of Deep Learning" 2019 [Workshop] Day 1 | Session 2De-Identifying Healthcare Data for ResearchIntelligent cloud computing lifts villages out of water povertyHow Microsoft and Novartis created Assess MS (short version)How OSIsoft and Deschutes Brewery used Microsoft Security Risk DetectionChronoZoom curriculum and technologyRecent Advances in Image Captioning, Image-Text Retrieval and…Using machine learning and AI to reduce hospital readmissionsIROS 2020 - Mixed Reality and Robotics Tutorial - Demo 1: InteractionCamera-based non-contact health sensing