Compound Emotion Recognition of Autistic Children During Meltdown Crisis Based on Deep Spatio-Temporal Analysis of Facial Geometric Features

• Main Authors: Salma Kammoun Jarraya, Marwa Masmoudi, Mohamed Hammami
• Format: Article
• Language: English
• Published: IEEE 2020-01-01
• Series: IEEE Access
• Subjects: Autism; deep spatio-temporal features; meltdown crisis; facial expressions; compound emotions
• Online Access: https://ieeexplore.ieee.org/document/9060825/

An important contribution to computer vision applications has been made by recognizing human emotion. Although it is very significant, this work considers the security of autistic people while in meltdown crisis by introducing a new system to warn caregivers through facial expressions detection. A precautionary approach has been taken to deal with meltdown crisis. Certainly, the indications of Meltdown are linked to abnormal facial expressions related to compound emotions. Actually, researchers thought long ago that Human Facial Expressions (HFE) are not able to express more than the seven basics emotions. HFE have been considered by psychologists as very complicated one, which can indicate two or even more emotions known as compound or mixed ones. A few studies have been done concerning Compound Emotion (CE). As well as, many difficult tasks to detect Compound Emotion Recognition (CER). In this paper, we empirically assess a group of deep spatio-temporal geometric features of micro-expressions of autistic children during a meltdown crisis. To achieve this goal, we make a comparison of the CER performance and diverse collections of micro-expressions features to select the features which best differentiates autistic children CE in meltdown crisis from normal state, and the best classifier performance. We record autistic children videos in normal and meltdown crisis using Kinect camera in serious circumstances. The experimental evaluation shows that the deep spatio-temporal geometric features and Recurrent Neural Network RNN with 3 hidden layer using Information Gain Feature Selection methods provide best performance (85.8%).