A Mobile Emotion Recognition System Based on Speech Signals and Facial Images

• Main Authors: Ivan, 吳于豪
• Other Authors: 楊東麟
• Format: Others
• Language: en_US
• Published: 2013
• Online Access: http://ndltd.ncl.edu.tw/handle/453f85

碩士 === 逢甲大學 === 資訊工程學系 === 101 === Smartphones are used for personal and business communications in daily life, and they have become a primary medium to capture human emotions. By recognizing the emotions of speakers during a conversation, one can deliver or understand messages better, make successful negotiations, and provide more personal services. Therefore, we developed an emotion recognition system on a mobile platform based on speech signals and facial images. This research has two phases, a training phase and a testing phase. In the training phase, speech signals and facial images are processed through data preprocessing, feature extraction, and SVM classifier training steps. In the testing phase, the participants generated video recordings as test data. These data were transformed for feature extraction and classified into four emotion classes by using the classifiers generated in the first phase. We adopted MFCC, DMFCC, and DDMFCC as the features of speech signals. Feature selection methods were exploited to choose useful features. We proposed an adjustable weighted segmentation method to determine the final results of emotion recognition. We performed various experiments with real world simulations to evaluate the proposed system. The results showed an average accuracy rate of 87 percent with the highest accuracy rate at 91 percent. Facial images were also used to improve emotion recognition especially during periods of silence in conversations.