Video Based Rendering and Coding using a Planar Patch Model

• Main Author: Bowen, Adam Mark Alexander
• Published: University of Warwick 2008
• Subjects: 621.36
• Online Access: http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.504799

Free viewpoint video is a rapidly evolving field with many challenging problems. The work in this thesis focuses on a number of these problems, from capture through to reconstruction, and aims to resolve them in a common 'patch based' framework. The patch based representation models scenes as a collection of disjoint, planar surface elements. This approach offers the advantages of continuous representations, such as geometric meshes, for reconstruction, and those of discontinuous representations, such as dense-depth maps, for tracking. Much like point based representations of geometry, patch based representations contain a large number of elements. In order to track and process such representations it is desirable to be able to model these data hierarchically. To this end, a novel bottom-up process is introduced for -modelling arbitrary objects as a hierarchical mixture of Gaussian densities. The approach is contrasted with some existing general purpose methods, and found to be superior for hierarchical object modelling. A reconstruction algorithm is introduced for computing high quality synthesised views from patch based representations. The proposed algorithm employs a local visual hull to overcome problems in representing object edges using patches. The results are examined over a variety of sequences, captured under a variety of scenarios. Quantitative analysis suggests the algorithm significantly outperforms simple approaches and is comparable with recently published approaches. In addition, a simple extension to MPEG-2 for the compression of multiview sequences is introduced and contrasted with state-of-the-art approaches. Although not competitive with approaches based upon H.264, the proposed method has the advantage of simplicity in implementation and execution. This is important when dealing with the substantial volumes of data present in multiview video sequences. Keywords: Image Based Rendering, Free Viewpoint Video, Hierarchical Clustering, Multiresolution Gaussian Mixture, Patch Based Representations.