Image-based deformable 3D reconstruction using differential geometry and cartan's connections

• Main Author: Parashar, Shaifali
• Other Authors: Clermont Auvergne
• Language: en
• Published: 2017
• Subjects: Vision par ordinateur; Reconstruction 3D; Géométrie différentielle; NRSfM; SfT; Cartan; Connexions; Computer vision; 3D reconstruction; Differential geometry; Connections
• Online Access: http://www.theses.fr/2017CLFAC078/document

La reconstruction 3D d’objets à partir de plusieurs images est un objectif important de la vision par ordinateur. Elle a été largement étudiée pour les objets rigides et non rigides (ou déformables). Le Structure-from-Motion (SfM) est un algorithme qui effectue la reconstruction 3D d’objets rigides en utilisant le mouvement visuel entre plusieurs images obtenues à l’aide d’une caméra en mouvement. Le SfM est une solution très précise et stable. La reconstruction 3D déformable a été largement étudiée pour les images monoculaires (obtenues à partir d’une seule caméra) mais reste un problème ouvert. Les méthodes actuelles exploitent des indices visuels tels que le mouvement visuel inter-image et l’ombrage afin de construire un algorithme de reconstruction. Cette thèse se concentre sur l’utilisation du mouvement visuel inter-image pour résoudre ce problème. Deux types de scénarios existent dans la littérature : 1) le Non-Rigid Structure-from-Motion (NRSfM) et 2) le Shape-from-Template (SfT). L’objectif du NRSfM est de reconstruire plusieurs formes d’un objet déformable tel qu’il apparaît dans plusieurs images, alors que le SfT (également appelé reconstruction à partir d’un modèle de référence) utilise une seule image d’un objet déformé et son modèle 3D de référence (une forme 3D texturée de l’objet dans une configuration) pour estimer la forme déformée de l’objet. (...) === Reconstructing the 3D shape of objects from multiple images is an important goal in computer vision and has been extensively studied for both rigid and non-rigid (or deformable) objects. Structure-from-Motion (SfM) is an algorithm that performs the 3D reconstruction of rigid objects using the inter-image visual motion from multiple images obtained from a moving camera. SfM is a very accurate and stable solution. Deformable 3D reconstruction, however, has been widely studied for monocular images (obtained from a single camera) and still remains an open research problem. The current methods exploit visual cues such as the inter-image visual motion and shading in order to formalise a reconstruction algorithm. This thesis focuses on the use of the inter-image visual motion for solving this problem. Two types of scenarios exist in the literature: 1) Non-Rigid Structure-from-Motion (NRSfM) and 2) Shape-from-Template (SfT). The goal of NRSfM is to reconstruct multiple shapes of a deformable object as viewed in multiple images while SfT (also referred to as template-based reconstruction) uses a single image of a deformed object and its 3D template (a textured 3D shape of the object in one configuration) to recover the deformed shape of the object. We propose an NRSfM method to reconstruct the deformable surfaces undergoing isometric deformations (the objects do not stretch or shrink under an isometric deformation) using Riemannian geometry. This allows NRSfM to be expressed in terms of Partial Differential Equations (PDE) and to be solved algebraically. We show that the problem has linear complexity and the reconstruction algorithm has a very low computational cost compared to existing NRSfM methods. This work motivated us to use differential geometry and Cartan’s theory of connections to model NRSfM, which led to the possibility of extending the solution to deformations other than isometry. In fact, this led to a unified theoretical framework for modelling and solving both NRSfM and SfT for various types of deformations. In addition, it also makes it possible to have a solution to SfT which does not require an explicit modelling of deformation. An important point is that most of the NRSfM and SfT methods reconstruct the thin-shell surface of the object. The reconstruction of the entire volume (the thin-shell surface and the interior) has not been explored yet. We propose the first SfT method that reconstructs the entire volume of a deformable object.