Highly Robust and Wearable Facial Expression Recognition via Deep-Learning-Assisted, Soft Epidermal Electronics
The facial expressions are a mirror of the elusive emotion hidden in the mind, and thus, capturing expressions is a crucial way of merging the inward world and virtual world. However, typical facial expression recognition (FER) systems are restricted by environments where faces must be clearly seen...
Main Authors: | , , , , , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
American Association for the Advancement of Science
2021-01-01
|
Series: | Research |
Online Access: | http://dx.doi.org/10.34133/2021/9759601 |
id |
doaj-cc18f2612ba74191829352e6946030d4 |
---|---|
record_format |
Article |
spelling |
doaj-cc18f2612ba74191829352e6946030d42021-07-26T08:39:19ZengAmerican Association for the Advancement of ScienceResearch2639-52742021-01-01202110.34133/2021/9759601Highly Robust and Wearable Facial Expression Recognition via Deep-Learning-Assisted, Soft Epidermal ElectronicsMeiqi Zhuang0Lang Yin1Lang Yin2Youhua Wang3Youhua Wang4Yunzhao Bai5Yunzhao Bai6Jian Zhan7Jian Zhan8Chao Hou9Chao Hou10Liting Yin11Liting Yin12Zhangyu Xu13Zhangyu Xu14Xiaohui Tan15YongAn Huang16YongAn Huang17Information Engineering College,Capital Normal University,Beijing 100048,ChinaState Key Laboratory of Digital Manufacturing Equipment and Technology,Huazhong University of Science and Technology,Wuhan 430074,ChinaFlexible Electronics Research Center,Huazhong University of Science and Technology,Wuhan 430074,ChinaState Key Laboratory of Digital Manufacturing Equipment and Technology,Huazhong University of Science and Technology,Wuhan 430074,ChinaFlexible Electronics Research Center,Huazhong University of Science and Technology,Wuhan 430074,ChinaState Key Laboratory of Digital Manufacturing Equipment and Technology,Huazhong University of Science and Technology,Wuhan 430074,ChinaFlexible Electronics Research Center,Huazhong University of Science and Technology,Wuhan 430074,ChinaState Key Laboratory of Digital Manufacturing Equipment and Technology,Huazhong University of Science and Technology,Wuhan 430074,ChinaFlexible Electronics Research Center,Huazhong University of Science and Technology,Wuhan 430074,ChinaState Key Laboratory of Digital Manufacturing Equipment and Technology,Huazhong University of Science and Technology,Wuhan 430074,ChinaFlexible Electronics Research Center,Huazhong University of Science and Technology,Wuhan 430074,ChinaState Key Laboratory of Digital Manufacturing Equipment and Technology,Huazhong University of Science and Technology,Wuhan 430074,ChinaFlexible Electronics Research Center,Huazhong University of Science and Technology,Wuhan 430074,ChinaState Key Laboratory of Digital Manufacturing Equipment and Technology,Huazhong University of Science and Technology,Wuhan 430074,ChinaFlexible Electronics Research Center,Huazhong University of Science and Technology,Wuhan 430074,ChinaInformation Engineering College,Capital Normal University,Beijing 100048,ChinaState Key Laboratory of Digital Manufacturing Equipment and Technology,Huazhong University of Science and Technology,Wuhan 430074,ChinaFlexible Electronics Research Center,Huazhong University of Science and Technology,Wuhan 430074,ChinaThe facial expressions are a mirror of the elusive emotion hidden in the mind, and thus, capturing expressions is a crucial way of merging the inward world and virtual world. However, typical facial expression recognition (FER) systems are restricted by environments where faces must be clearly seen for computer vision, or rigid devices that are not suitable for the time-dynamic, curvilinear faces. Here, we present a robust, highly wearable FER system that is based on deep-learning-assisted, soft epidermal electronics. The epidermal electronics that can fully conform on faces enable high-fidelity biosignal acquisition without hindering spontaneous facial expressions, releasing the constraint of movement, space, and light. The deep learning method can significantly enhance the recognition accuracy of facial expression types and intensities based on a small sample. The proposed wearable FER system is superior for wide applicability and high accuracy. The FER system is suitable for the individual and shows essential robustness to different light, occlusion, and various face poses. It is totally different from but complementary to the computer vision technology that is merely suitable for simultaneous FER of multiple individuals in a specific place. This wearable FER system is successfully applied to human-avatar emotion interaction and verbal communication disambiguation in a real-life environment, enabling promising human-computer interaction applications.http://dx.doi.org/10.34133/2021/9759601 |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Meiqi Zhuang Lang Yin Lang Yin Youhua Wang Youhua Wang Yunzhao Bai Yunzhao Bai Jian Zhan Jian Zhan Chao Hou Chao Hou Liting Yin Liting Yin Zhangyu Xu Zhangyu Xu Xiaohui Tan YongAn Huang YongAn Huang |
spellingShingle |
Meiqi Zhuang Lang Yin Lang Yin Youhua Wang Youhua Wang Yunzhao Bai Yunzhao Bai Jian Zhan Jian Zhan Chao Hou Chao Hou Liting Yin Liting Yin Zhangyu Xu Zhangyu Xu Xiaohui Tan YongAn Huang YongAn Huang Highly Robust and Wearable Facial Expression Recognition via Deep-Learning-Assisted, Soft Epidermal Electronics Research |
author_facet |
Meiqi Zhuang Lang Yin Lang Yin Youhua Wang Youhua Wang Yunzhao Bai Yunzhao Bai Jian Zhan Jian Zhan Chao Hou Chao Hou Liting Yin Liting Yin Zhangyu Xu Zhangyu Xu Xiaohui Tan YongAn Huang YongAn Huang |
author_sort |
Meiqi Zhuang |
title |
Highly Robust and Wearable Facial Expression Recognition via Deep-Learning-Assisted, Soft Epidermal Electronics |
title_short |
Highly Robust and Wearable Facial Expression Recognition via Deep-Learning-Assisted, Soft Epidermal Electronics |
title_full |
Highly Robust and Wearable Facial Expression Recognition via Deep-Learning-Assisted, Soft Epidermal Electronics |
title_fullStr |
Highly Robust and Wearable Facial Expression Recognition via Deep-Learning-Assisted, Soft Epidermal Electronics |
title_full_unstemmed |
Highly Robust and Wearable Facial Expression Recognition via Deep-Learning-Assisted, Soft Epidermal Electronics |
title_sort |
highly robust and wearable facial expression recognition via deep-learning-assisted, soft epidermal electronics |
publisher |
American Association for the Advancement of Science |
series |
Research |
issn |
2639-5274 |
publishDate |
2021-01-01 |
description |
The facial expressions are a mirror of the elusive emotion hidden in the mind, and thus, capturing expressions is a crucial way of merging the inward world and virtual world. However, typical facial expression recognition (FER) systems are restricted by environments where faces must be clearly seen for computer vision, or rigid devices that are not suitable for the time-dynamic, curvilinear faces. Here, we present a robust, highly wearable FER system that is based on deep-learning-assisted, soft epidermal electronics. The epidermal electronics that can fully conform on faces enable high-fidelity biosignal acquisition without hindering spontaneous facial expressions, releasing the constraint of movement, space, and light. The deep learning method can significantly enhance the recognition accuracy of facial expression types and intensities based on a small sample. The proposed wearable FER system is superior for wide applicability and high accuracy. The FER system is suitable for the individual and shows essential robustness to different light, occlusion, and various face poses. It is totally different from but complementary to the computer vision technology that is merely suitable for simultaneous FER of multiple individuals in a specific place. This wearable FER system is successfully applied to human-avatar emotion interaction and verbal communication disambiguation in a real-life environment, enabling promising human-computer interaction applications. |
url |
http://dx.doi.org/10.34133/2021/9759601 |
work_keys_str_mv |
AT meiqizhuang highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics AT langyin highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics AT langyin highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics AT youhuawang highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics AT youhuawang highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics AT yunzhaobai highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics AT yunzhaobai highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics AT jianzhan highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics AT jianzhan highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics AT chaohou highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics AT chaohou highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics AT litingyin highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics AT litingyin highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics AT zhangyuxu highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics AT zhangyuxu highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics AT xiaohuitan highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics AT yonganhuang highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics AT yonganhuang highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics |
_version_ |
1721282139280900096 |