The Effect of Beautification Filters on Image Recognition : "Are filtered social media images viable Open Source Intelligence?"

• Main Authors: Skepetzis, Vasilios, Hedman, Pontus
• Format: Others
• Language: English
• Published: Högskolan i Halmstad, Akademin för informationsteknologi 2021
• Subjects: face recognition; OSINT; machine learning; deep learning; convolutional neural networks; social media filters; u-net; residual neural network; Ansiktsigenkänning; maskininlärning; djupinlärning; faltningsnätverk; sociala media filter; residual neuronnät; Computer Vision and Robotics (Autonomous Systems); Datorseende och robotik (autonoma system); Computer and Information Sciences; Data- och informationsvetenskap; Signal Processing; Signalbehandling; Computer Systems; Datorsystem
• Online Access: http://urn.kb.se/resolve?urn=urn:nbn:se:hh:diva-44799

In light of the emergence of social media, and its abundance of facial imagery, facial recognition finds itself useful from an Open Source Intelligence standpoint. Images uploaded on social media are likely to be filtered, which can destroy or modify biometric features. This study looks at the recognition effort of identifying individuals based on their facial image after filters have been applied to the image. The social media image filters studied occlude parts of the nose and eyes, with a particular interest in filters occluding the eye region. Our proposed method uses a Residual Neural Network Model to extract features from images, with recognition of individuals based on distance measures, based on the extracted features. Classification of individuals is also further done by the use of a Linear Support Vector Machine and XGBoost classifier. In attempts to increase the recognition performance for images completely occluded in the eye region, we present a method to reconstruct this information by using a variation of a U-Net, and from the classification perspective, we also train the classifier on filtered images to increase the performance of recognition. Our experimental results showed good recognition of individuals when filters were not occluding important landmarks, especially around the eye region. Our proposed solution shows an ability to mitigate the occlusion done by filters through either reconstruction or training on manipulated images, in some cases, with an increase in the classifier’s accuracy of approximately 17% points with only reconstruction, 16% points when the classifier trained on filtered data, and  24% points when both were used at the same time. When training on filtered images, we observe an average increase in performance, across all datasets, of 9.7% points.