A Modular System for Detection, Tracking and Analysis of Human Faces in Thermal Infrared Recordings

• Main Authors: Marcin Kopaczka, Lukas Breuer, Justus Schock, Dorit Merhof
• Format: Article
• Language: English
• Published: MDPI AG 2019-09-01
• Series: Sensors
• Subjects: thermal infrared imaging; image processing; face detection; face analysis
• Online Access: https://www.mdpi.com/1424-8220/19/19/4135

We present a system that utilizes a range of image processing algorithms to allow fully automated thermal face analysis under both laboratory and real-world conditions. We implement methods for face detection, facial landmark detection, face frontalization and analysis, combining all of these into a fully automated workflow. The system is fully modular and allows implementing own additional algorithms for improved performance or specialized tasks. Our suggested pipeline contains a histogtam of oriented gradients support vector machine (HOG-SVM) based face detector and different landmark detecion methods implemented using feature-based active appearance models, deep alignment networks and a deep shape regression network. Face frontalization is achieved by utilizing piecewise affine transformations. For the final analysis, we present an emotion recognition system that utilizes HOG features and a random forest classifier and a respiratory rate analysis module that computes average temperatures from an automatically detected region of interest. Results show that our combined system achieves a performance which is comparable to current stand-alone state-of-the-art methods for thermal face and landmark datection and a classification accuracy of 65.75% for four basic emotions.