MUTUAL IMAGE TRANSFORMATION ALGORITHMS FOR VISUAL INFORMATION PROCESSING AND RETRIEVAL

• Main Authors: G. A. Kukharev, Y. N. Matveev, A. L. Oleinik
• Format: Article
• Language: English
• Published: Saint Petersburg National Research University of Information Technologies, Mechanics and Optics (ITMO University) 2017-01-01
• Series: Naučno-tehničeskij Vestnik Informacionnyh Tehnologij, Mehaniki i Optiki
• Subjects: cross-modal multimedia retrieval; principal component analysis; face images; sketch; facial composite
• Online Access: http://ntv.ifmo.ru/file/article/16407.pdf

Subject of Research. The paper deals with methods and algorithms for mutual transformation of related pairs of images in order to enhance the capabilities of cross-modal multimedia retrieval (CMMR) technologies. We have thoroughly studied the problem of mutual transformation of face images of various kinds (e.g. photos and drawn pictures). This problem is widely represented in practice. Research is this area is based on existing datasets. The algorithms we have proposed in this paper can be applied to arbitrary pairs of related images due to the unified mathematical specification. Method. We have presented three image transformation algorithms. The first one is based on principal component analysis and Karhunen-Loève transform (1DPCA/1DKLT). Unlike the existing solution, it does not use the training set during the transformation process. The second algorithm assumes generation of an image population. The third algorithm performs the transformation based on two-dimensional principal component analysis and Karhunen-Loève transform (2DPCA/2DKLT). Main Results. The experiments on image transformation and population generation have revealed the main features of each algorithm. The first algorithm allows construction of an accurate and stable model of transition between two given sets of images. The second algorithm can be used to add new images to existing bases and the third algorithm is capable of performing the transformation outside the training dataset. Practical Relevance. Taking into account the qualities of the proposed algorithms, we have provided recommendations concerning their application. Possible scenarios include construction of a transition model for related pairs of images, mutual transformation of the images inside and outside the dataset as well as population generation in order to increase representativeness of existing datasets. Thus, the proposed algorithms can be used to improve reliability of face recognition performed on images of various kinds. Moreover, these techniques can be applied to address a wide variety of other CMMR problems.