A Study on Single/Multiple Sprite Generation and Partition for Videos

• Main Authors: I-Sheng Kuo, 郭萓聖
• Other Authors: Ling-Hwei Chen
• Format: Others
• Language: en_US
• Published: 2008
• Online Access: http://ndltd.ncl.edu.tw/handle/17877468659347860453

博士 === 國立交通大學 === 資訊科學與工程研究所 === 96 === Sprite coding, which can increase the coding efficiency of backgrounds greatly, is a novel technology adopted in MPEG-4 object-based coding. The sprite generator introduced in MPEG-4 blends frames by averaging blending, this will make some places, which are ever occupied by moving objects, look blurring. Thus, providing segmented masks for moving objects is suggested. We build a sprite generation system based on MPEG-4’s framework, but we find that using manual segmentation masks in a sprite generation system is impractical. An automatic segmentation mask generation method is proposed and is applied in the sprite generation system. The sprite generation system produces a coarse sprite first by MPEG-4’s method without segmentation masks. Then the coarse sprite is employed as the reference image in the proposed segmentation mask generation method. After generating the segmentation masks, a better sprite is re-generated again with generated segmentation masks. Experimental results show the sprite generated by the proposed system has good quality. Automatic image segmentation can not produce perfect object segmentation masks. Segmentation faults in segmentation masks causes some moving objects being blended into a sprite. This makes some ghost-like shadows appear in a generated sprite. To treat this problem, a sprite generation without segmentation masks is proposed in this dissertation. The proposed sprite generator consists of two novel methods: a balanced feature point extraction method and an intelligent blending method. The feature point extraction method estimates the motion vector of background pixels, and excludes pixels of moving objects from the feature points. Proposed intelligent blending method blends only background pixels into a sprite by a simple counting schema. Experimental results show the feature points extracted by the proposed method increases the accuracy of global motion estimation, and the quality of generated sprites is increased. The proposed intelligent blending method excludes pixels of moving objects directly in the blending procedure. Thus ghost-like shadows caused by segmentation faults is not exist in the sprite generated by our method. The visual quality of our sprite is close to that using manually segmented masks and is better than that generated by Smolic et al.’s method. Due to the geometric transformation applied to each non-reference frame in the procedure of sprite coding, the generated sprite is distorted and the available view angles relative to the reference frame are restricted. This makes multiple sprites used be necessary. An optimal multiple sprite generation method has been proposed by Farin et al., but it uses an exhaustive search to find the optimal partition and reference frames. Let N be the number of frames, Frains’ method requires O(N3) time and O(N2) space to perform the search. In order to reduce the complexity, a fast multiple sprite partition method is proposed in this dissertation. The proposed method includes a fast partition point finding method and a fast reference frame finding method. The proposed partition point finding method measures translation and scaling between frames and finds candidate partition points by the measured values. The final partition positions are decided from these candidate points, and reference frames of each partition are found by the proposed fast reference frame selecting method. Let M candidate partition points are found, the proposed method requires only O(M2N) in time and O(M2)+O(N) in space. The total size of generated sprites is only slightly higher than that of Farin’s method.