Fast Algorithm and Hardware Architecture for Stereo Image Synthesis Systems

• Main Authors: Wan-Yu Chen, 陳菀瑜
• Other Authors: 陳良基
• Format: Others
• Language: en_US
• Published: 2006
• Online Access: http://ndltd.ncl.edu.tw/handle/73702757810501402608

碩士 === 國立臺灣大學 === 電子工程學研究所 === 94 === Stereo images can make users sense depth perception by showing two images to each eye simultaneously. It gives users a vivid information about the scene structure and can be used for 3D-TV, telepresence, and immersive communication. With the technologies of 3D video capture device and 3D-TV display device getting more and more mature, the importance of the stereo content will rise in the near future. Under this trend, stereo image synthesis draws more and more attention. Stereo image synthesis is the essential part in 3D image system. It is also the main part which is focused on in this thesis. However, to build up a stereo image synthesis system, many design challenges, such as bad synthesis quality, high computational complexity, and hardware architecture implementation, etc., must be overcome. In this thesis,a novel stereo image synthesis system from single lens auto focus camera is proposed to overcome the overheads of traditional two lens stereo camera. First we proposed a novel object based depth estimation algorithm to estimate depth map from multi-focus images. Besides, we adopted depth-image-based-rendering with edge dependent gaussian filter and interpolation to render left and right image with high quality. Finally, we proposed a new depth background mode to enrich the background depth. The proposed single camera stereo image synthesis system with efficient object based depth from focus and depth image based rendering core design is proposed and discussed in algorithm level, architecture level, and system level. In algorithm level, first we focus our attention on the first stage: Object-Based-Depth-From-Focus(OBDFF). We proposed a new object based DFF to increase the noise resistibility and discriminability of depth map with multi-objects. Besides, the proposed algorithm only require less than ten multi-focus images to accomplish stereo image generation. The simulation results show the proposed object based depth from focus method can reach more than 95% match rate compared with ground truth depth map. Furthermore, the novel color segmentation based depth interpolation reduced the run-time of depth interpolation in depth from focus by 99.6%. Then we focus our attention on the second stage: Efficient Depth Image Based Rendering(DIBR) with Edge Dependent Gaussian Filter and Interpolation The proposed DIBR algorithm outperforms previous work by 6–7 dB in PSNR performance. In addition to that, the number of instruction cycles is 3.7 percent compared with the previous work. Besides, in architecture level, a hardwired DIBR core architecture design is proposed to provide a cost-effective solution to DIBR core implementation in not only stereo image synthesis system but also advanced threedimensional television system. We propose a novel depth image based rendering algorithm with edge-dependent gaussian filter and interpolation to improve the rendered stereo image quality. Based on our proposed algorithm, a fully-pipelined depth image based rendering hardware accelerator is proposed to support real-time rendering. The proposed hardware accelerator is optimized in three steps. First, we analyze the effect of fixed point operation and choose the optimal wordlength to keep the stereo image quality. Second, a three-parallel edge-dependent gaussian filter architecture is proposed to solve the critical problem of memory bandwidth. Finally, we optimize the hardware cost by the proposed hardware architecture. Only 1/21 amounts of vertical PEs and 1/11 amounts of horizontal PEs is needed by the proposed folded edge-dependent gaussian filter architecture. Futhermore, by the proposed check mode, the whole Z-buffer can be eliminated during 3D image warping. In additions, the on-chip SRAMs can be reduced to 66.7 percent compared with direct implementation by global and local disparity separation scheme. A prototype chip can achieve real-time requirement under the operating frequency of 80 MHz for 25 SDTV frames per second (fps) in left and right channel simultaneously. The simulation result also shows the hardware cost is quite small compared with the conventional rendering architecture. Moreover, in system integration level, a prototype of real-time stereo image synthesis system is successfully built up. The system integrates a single camera, FPGA accelerator, software optimization, and autostereoscopic 3D display LCD. The demonstration results proves proposed algorithm and architecture of the DIBR core indeed improve the performance in the stereo image synthesis system.