The Study on Recognizing Learning Emotion Based on Convolutional Neural Networks and Transfer Learning

• Main Authors: Nian-Xiang Lai, 賴念翔
• Other Authors: Kuan-Cheng Lin
• Format: Others
• Language: zh-TW
• Published: 2018
• Online Access: http://ndltd.ncl.edu.tw/handle/twdr5f

碩士 === 國立中興大學 === 資訊管理學系所 === 106 === In classroom teaching, if teachers want to understand the learning effectiveness of learners, they often collect and analyze data through quizzes or questionnaires, but they can’t receive real-time feedback. The learner’s facial emotions are highly correlated with learning motivation and effectiveness. Recognizing the learner’s facial emotions through the system helps the learners to understand the learning situations of themselves and make teachers provide help and improvement in class teaching. Research indicates that the Convolution Neural Network (CNN) in basic emotions face recognition has a good performance. Because CNN do not require hand-designed features like traditional machine learning, they automatically learn the necessary features of the entire image. We improves the CNN architecture FaceLiveNet which has low parameter and high accuracy in basic emotion recognition, and proposes Dense_FaceLiveNet architecture. We use Dense_FaceLiveNet for two-phases of transfer learning. First, from the relatively simple data JAFFE and KDEF basic emotion recognition model transferring to the FER2013 basic emotion dataset and obtained an accuracy of 70.02%. Secondly, using the FER2013 basic emotion recognition model transferring to learning emotion recognition model, the test accuracy rate is as high as 91.93%, which is 12.03% higher than the accuracy rate of 79.03% without using the transfer learning model, which proves that the use of transfer learning can Effectively improve the recognition accuracy of learning emotion recognition model. In addition, in order to test the generalization ability of the learning emotion recognition model, videos recorded by students from a national university in Taiwan during class learning were used as test data. The original database of learning emotions did not consider that students would have exceptions such as over eyebrows, eyes closed and hand hold the chin etc. To improve this situation, after adding the learning emotion database to the images of the exceptions mentioned above, the model was re-build, and the recognition accuracy rate of the model was 91.42%. By comparing the output of the maps, the model does have the characteristics of success in learning images such as eyebrows, chins, and eyes closed. Further, after combining all the students'' image data with the original learning emotion database, the model was re-build and obtained the accuracy rate reached 84.59%. The result proves that the learning emotion recognition model can achieve high recognition accuracy by processing the unlearned image through transfer learning.