Deformable Surface 3D Reconstruction from Monocular Images

Being able to recover the shape of 3D deformable surfaces from a single video stream would make it possible to field reconstruction systems that run on widely available hardware without requiring specialized devices. However, because many different 3D shapes can have virtually the same projection, such monocular shape recovery is inherently ambiguous. In this survey, we will review the two main classes of techniques that have proved most effective so far: The template-based methods that rely on establishing correspondences with a reference image in which the shape is already known, and non-rigid structure-from-motion techniques that exploit points tracked across the sequences to reconstruct a completely unknown shape. In both cases, we will formalize the approach, discuss its inherent ambiguities, and present the practical solutions that have been proposed to resolve them. To conclude, we will suggest directions for future research. Table of Contents: Introduction / Early Approaches toNon-Rigid Reconstruction / Formalizing Template-Based Reconstruction / Performing Template-Based Reconstruction / Formalizing Non-Rigid Structure from Motion / Performing Non-Rigid Structure from Motion / Future Directions

Autorentext

Gabriela Csurka is a Principal Scientist at NAVER LABS Europe, France. Her main research interests are in computer vision for image understanding, 3D reconstruction, visual localization, as well as domain adaptation and transfer learning. She has contributed to around 100 scientific communications, several on the topic of DA. She has given several invited talks and organized a tutorial on domain adaptation at ECCV'20. In 2017 she edited the Springer book Domain Adaptation for Computer Vision Applications. Timothy M. Hospedales is a Professor at the University of Edinburgh; Principal Researcher at Samsung AI Research Centre, Cambridge; and Alan Turing Institute Fellow. His research focuses on lifelong machine learning, broadly defined to include multi-domain/multi-task learning, domain adaptation, transfer learning, and meta-learning, with applications including computer vision, language, reinforcement learning for control, and finance. He has co-authored numerous papers on domain adaptation, domain generalization, and transfer learning in major venues including CVPR, ICCV, ECCV, ICML, ICLR, NeurIPS, and AAAI. He teaches computer vision at Edinburgh University and has given invited talks and tutorials on these topics at various international venues, renowned universities, and research institutes. Mathieu Salzmann is a Senior Researcher at EPFL and, since May 2020, a part-time Artificial Intelligence Engineer at ClearSpace. His research focuses on developing machine learning algorithms for visual scene understanding, including object recognition, detection, semantic segmentation, 6D pose estimation, and 3D reconstruction. He has published articles on the topic of domain adaptation at major venues, including CVPR, ICCV, ICLR, AAAI, TPAMI, and JMLR. Furthermore, he has been invited to present his domain adaptation work at various venues and internationally renowned universities. Tatiana Tommasi is Associate Professor at Politecnico di Torino, Italy and an affiliated researcher at the Italian Institute of Technology. She pioneered the area of transfer learning for computer vision and has large experience in domain adaptation, generalization, and multimodal learning with applications for robotics and medical imaging. Tatiana received the best paper award at the 1st edition of Task-CV workshop at ECCV'14 and since then she has been leading the organization of the following workshop editions. She also organized a workshop on similar topics at NIPS'13 and '14 and taught a tutorial at ECCV'14 and '20.

Inhalt
Introduction.- Early Approaches to Non-Rigid Reconstruction.- Formalizing Template-Based Reconstruction.- Performing Template-Based Reconstruction.- Formalizing Non-Rigid Structure from Motion.- Performing Non-Rigid Structure from Motion.- Future Directions.