Image/Video Super-Resolution and Fusion Using Visual Attention Model, Artificial Neural Network, and Particle Swarm Optimization

• Main Authors: Chen, Hsuan-Ying, 陳軒盈
• Other Authors: Leou, Jin-Jang
• Format: Others
• Language: en_US
• Published: 2011
• Online Access: http://ndltd.ncl.edu.tw/handle/14706893808993433668

博士 === 國立中正大學 === 資訊工程研究所 === 99 === Human perception tends to firstly pick attended regions, which correspond to prominent objects in an image. Visual attention region detection simulates the behavior of the human visual system (HVS) and detects regions of interest (ROIs) in the image. Artificial neural network (ANN) is a biologically motivated learning machine, which simulates the structure and behavior of the nervous system. The mathematical model comprises individual processing units called neurons that resemble neural activity. ANN is a powerful tool for dealing with nonlinearities. Particle swarm optimization (PSO), a population-based optimization algorithm, belongs to an evolutionary computation paradigm. It is an off-line optimization and suitable for solving a complex problem at low cost. In this thesis, first, a visual attention region detection approach using low-level texture and object features is addressed. The new and improved (shifted) functions are proposed and used in both the proposed texture and object features to ensure that all attended pixels will be extracted. The proposed approach can generate high-quality spatial saliency maps in an effective manner. Second, a saliency-directed image interpolation approach using PSO is addressed. A block-based saliency map of an image to be interpolated is generated by the modified visual attention model in an effective manner. Then, based on the block-based saliency map, bilinear interpolation and PSO interpolation are employed for the pixels in “non-saliency” blocks and “saliency” blocks, respectively, to obtain the final interpolation results. Third, a saliency-directed color image interpolation approach using ANN and PSO is addressed. A high-quality saliency map of a color image to be interpolated is generated by the modified block-based visual attention model in an effective manner. Then, based on the saliency map, bilinear interpolation and ANN-PSO interpolation are employed for “non-saliency” blocks (non-ROIs) and “saliency” blocks (ROIs), respectively, to obtain the final color interpolation results. Fourth, a learning-based video super-resolution (SR) reconstruction approach using PSO is proposed. A motion-compensated volume containing five motion-compensated patches and the edge orientation of the volume are extracted and determined, respectively, for each pixel in the “central” reference low-resolution (LR) video frame. Then, the pixel values of the “central” reference high-resolution (HR) video frame are reconstructed by using the corresponding SR reconstruction filtering masks, based on the volume edge orientations and the coordinates of the pixels to be reconstructed. Fifth, a multispectral and multiresolution image fusion approach using PSO is proposed. The pixels of fused images in the training set are classified into several categories based on the characteristics of LR multispectral (MS) images. Then, the smooth parameters of spatial and spectral responses between the HR panchromatic (PAN) and LR MS images are determined by PSO. All the pixels within each category are normalized by its own smooth parameter.