Method of Facial Expression Analysis Using Video Phone and Thermal Image

• Main Authors: Yasunari Yoshitomi, Taro Asada, Ryota Kato, Masayoshi Tabuse
• Format: Article
• Language: English
• Published: Atlantis Press 2014-06-01
• Series: Journal of Robotics, Networking and Artificial Life (JRNAL)
• Subjects: Facial expression analysis; Video phone; Area of mouth and jaw; Thermal image; and Skype
• Online Access: https://www.atlantis-press.com/article/13175.pdf

To improve the quality of life of elderly people living in a home or healthcare facility, especially in a rural area, we have been developing a method for analyzing the facial expressions of a person using a video phone system (Skype) to speak with another person. In the present study, we proposed a method for analyzing facial expressions of a person using the video phone system to talk to another person. The recorded video is analyzed by thermal image processing and the newly proposed feature vector of facial expression, which is extracted in the mouth area by applying 2D-DCT. The facial expression intensity, defined as the norm of the difference vector between the feature vector of the neutral facial expression and that of the observed expression, can be used to analyze a change of facial expression. The judgment of utterance is performed by using the intensity of the sound wave. The experimental results show the usefulness of the proposed method. We intend to use the proposed method of facial expression analysis to develop a method for estimating the emotions or mental state of people, especially elderly patients.