| Summary: | This thesis investigates the use of robust deformable model for 3D face alignment and tracking. Our main objective is to establish accurate and reliable dense correspondence for 4D faces. Our contribution is two-fold. First, we build a robust face alignment framework to establish dense correspondence between 4D faces. Second, we develop 4DFAB, the first large scale high-resolution 4D face database for facial expression analysis and biometric. Our deformable 4D face alignment framework contains three parts: (1) robust 2D face alignment; (2) active non-rigid 3D face registration; (3) deformable 4D face tracking. For 2D face alignment, we start with the study of robust 2D and 3D geometry features (and their fusion) for Constrained Local Models (CLMs). We show that by leveraging robust features, CLMs can handle faces captured in controlled and uncontrolled environment. To exploit the discriminative power of CLMs, we propose an alignment framework based on the texture model of response maps. Under this framework, we devise two generative fitting methods (GFRM-Alt and GFRM-PO) and one part-based discriminative fitting method (DFRM), which achieve favorable performances in generic face alignment in-the-wild. Additionally, we implement a real-time face tracking software using DFRM. Next, we study the registration problem for high quality 3D facial scan. We build a part-based statistical face model and combine it with the non-rigid ICP. We name the method as Active Non-rigid ICP (ANICP). ANICP is integrated into a dynamic local fitting framework and produces accurate fitting. Note that DFRM is also used to provide better initialisation. Finally, we develop a dense 4D alignment framework that capitalises on GRMF-Alt and ANICP. This framework is employed to align faces from 4DFAB database, which contains over 1,800,000 meshes from 180 subjects captured in four different sessions during 5 years. As the subjects display both spontaneous and posed expressions, it can also be used for facial expression modeling and analysis. To demonstrate the usefulness of 4DFAB, we conduct extensive expression recognition, face recognition, speech recognition experiments. We also investigate, for the first time, the use of spontaneous 4D behaviour for biometric applications.
|
|---|
