Real time sub-pixel space-time stereo on the GPU

• Main Author: Nahmias, J.
• Published: University College London (University of London) 2013
• Subjects: 004
• Online Access: http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.587770

Recent advances in virtual reality, 3d computer generated graphics and computer vision are making the goal of producing a compelling interactive 3d face to face communication system more tractable. The problem with producing such a system is reconstructing the 3d geometry of the users in real-time. There are many ways of tackling this problem however many of them require prior knowledge (i.e model fitting methods). These add unnecessary constraints and limit the usability of the system to reconstructing known entities. Other high quality methods using laser triangulation require too many samples and therefore cannot handle dynamic and deformable shapes such as the human face. A more suited approach is to use stereo based algorithm that function using two of more views and augmenting their capabilities using structured light. The work presented in this thesis will examine and evaluate various stereo vision algorithms and hybrids with the goal of producing accurate 3d representations of human faces in real time. Various dynamic programming algorithms will be presented and hybrid variations. These will be extended into the space-time domain and the impact of using different structured light patterns with various algorithms and cost functions will be examined. Most real-time correspondence algorithms are limited to producing pixel value disparities; these can be augmented into producing sub-pixel disparities by smoothing functions. Applying such smoothing functions tends to remove detail. Another approach is to use non-linear optimization on a spatial-temporal warp function. These algorithms tend to be very computationally expensive and therefore not feasible for real time applications. With recent development of GPUs (Graphics Processing Units) driven by the consumer demand for complex real time 3d graphics, these cards are capable of processing large amounts of data in parallel. This makes them very amenable to solving large linear algebra problems. . The result being a tuneable stereo reconstruction framework that has been reformulated into streaming problems in order to be processed on the GPU to produce real time sub-pixel depth maps of human faces that can be triangulated to produce accurate 3d models.