Highly Robust and Wearable Facial Expression Recognition via Deep-Learning-Assisted, Soft Epidermal Electronics

The facial expressions are a mirror of the elusive emotion hidden in the mind, and thus, capturing expressions is a crucial way of merging the inward world and virtual world. However, typical facial expression recognition (FER) systems are restricted by environments where faces must be clearly seen...

Full description

Bibliographic Details
Main Authors: Meiqi Zhuang, Lang Yin, Youhua Wang, Yunzhao Bai, Jian Zhan, Chao Hou, Liting Yin, Zhangyu Xu, Xiaohui Tan, YongAn Huang
Format: Article
Language:English
Published: American Association for the Advancement of Science 2021-01-01
Series:Research
Online Access:http://dx.doi.org/10.34133/2021/9759601
id doaj-cc18f2612ba74191829352e6946030d4
record_format Article
spelling doaj-cc18f2612ba74191829352e6946030d42021-07-26T08:39:19ZengAmerican Association for the Advancement of ScienceResearch2639-52742021-01-01202110.34133/2021/9759601Highly Robust and Wearable Facial Expression Recognition via Deep-Learning-Assisted, Soft Epidermal ElectronicsMeiqi Zhuang0Lang Yin1Lang Yin2Youhua Wang3Youhua Wang4Yunzhao Bai5Yunzhao Bai6Jian Zhan7Jian Zhan8Chao Hou9Chao Hou10Liting Yin11Liting Yin12Zhangyu Xu13Zhangyu Xu14Xiaohui Tan15YongAn Huang16YongAn Huang17Information Engineering College,Capital Normal University,Beijing 100048,ChinaState Key Laboratory of Digital Manufacturing Equipment and Technology,Huazhong University of Science and Technology,Wuhan 430074,ChinaFlexible Electronics Research Center,Huazhong University of Science and Technology,Wuhan 430074,ChinaState Key Laboratory of Digital Manufacturing Equipment and Technology,Huazhong University of Science and Technology,Wuhan 430074,ChinaFlexible Electronics Research Center,Huazhong University of Science and Technology,Wuhan 430074,ChinaState Key Laboratory of Digital Manufacturing Equipment and Technology,Huazhong University of Science and Technology,Wuhan 430074,ChinaFlexible Electronics Research Center,Huazhong University of Science and Technology,Wuhan 430074,ChinaState Key Laboratory of Digital Manufacturing Equipment and Technology,Huazhong University of Science and Technology,Wuhan 430074,ChinaFlexible Electronics Research Center,Huazhong University of Science and Technology,Wuhan 430074,ChinaState Key Laboratory of Digital Manufacturing Equipment and Technology,Huazhong University of Science and Technology,Wuhan 430074,ChinaFlexible Electronics Research Center,Huazhong University of Science and Technology,Wuhan 430074,ChinaState Key Laboratory of Digital Manufacturing Equipment and Technology,Huazhong University of Science and Technology,Wuhan 430074,ChinaFlexible Electronics Research Center,Huazhong University of Science and Technology,Wuhan 430074,ChinaState Key Laboratory of Digital Manufacturing Equipment and Technology,Huazhong University of Science and Technology,Wuhan 430074,ChinaFlexible Electronics Research Center,Huazhong University of Science and Technology,Wuhan 430074,ChinaInformation Engineering College,Capital Normal University,Beijing 100048,ChinaState Key Laboratory of Digital Manufacturing Equipment and Technology,Huazhong University of Science and Technology,Wuhan 430074,ChinaFlexible Electronics Research Center,Huazhong University of Science and Technology,Wuhan 430074,ChinaThe facial expressions are a mirror of the elusive emotion hidden in the mind, and thus, capturing expressions is a crucial way of merging the inward world and virtual world. However, typical facial expression recognition (FER) systems are restricted by environments where faces must be clearly seen for computer vision, or rigid devices that are not suitable for the time-dynamic, curvilinear faces. Here, we present a robust, highly wearable FER system that is based on deep-learning-assisted, soft epidermal electronics. The epidermal electronics that can fully conform on faces enable high-fidelity biosignal acquisition without hindering spontaneous facial expressions, releasing the constraint of movement, space, and light. The deep learning method can significantly enhance the recognition accuracy of facial expression types and intensities based on a small sample. The proposed wearable FER system is superior for wide applicability and high accuracy. The FER system is suitable for the individual and shows essential robustness to different light, occlusion, and various face poses. It is totally different from but complementary to the computer vision technology that is merely suitable for simultaneous FER of multiple individuals in a specific place. This wearable FER system is successfully applied to human-avatar emotion interaction and verbal communication disambiguation in a real-life environment, enabling promising human-computer interaction applications.http://dx.doi.org/10.34133/2021/9759601
collection DOAJ
language English
format Article
sources DOAJ
author Meiqi Zhuang
Lang Yin
Lang Yin
Youhua Wang
Youhua Wang
Yunzhao Bai
Yunzhao Bai
Jian Zhan
Jian Zhan
Chao Hou
Chao Hou
Liting Yin
Liting Yin
Zhangyu Xu
Zhangyu Xu
Xiaohui Tan
YongAn Huang
YongAn Huang
spellingShingle Meiqi Zhuang
Lang Yin
Lang Yin
Youhua Wang
Youhua Wang
Yunzhao Bai
Yunzhao Bai
Jian Zhan
Jian Zhan
Chao Hou
Chao Hou
Liting Yin
Liting Yin
Zhangyu Xu
Zhangyu Xu
Xiaohui Tan
YongAn Huang
YongAn Huang
Highly Robust and Wearable Facial Expression Recognition via Deep-Learning-Assisted, Soft Epidermal Electronics
Research
author_facet Meiqi Zhuang
Lang Yin
Lang Yin
Youhua Wang
Youhua Wang
Yunzhao Bai
Yunzhao Bai
Jian Zhan
Jian Zhan
Chao Hou
Chao Hou
Liting Yin
Liting Yin
Zhangyu Xu
Zhangyu Xu
Xiaohui Tan
YongAn Huang
YongAn Huang
author_sort Meiqi Zhuang
title Highly Robust and Wearable Facial Expression Recognition via Deep-Learning-Assisted, Soft Epidermal Electronics
title_short Highly Robust and Wearable Facial Expression Recognition via Deep-Learning-Assisted, Soft Epidermal Electronics
title_full Highly Robust and Wearable Facial Expression Recognition via Deep-Learning-Assisted, Soft Epidermal Electronics
title_fullStr Highly Robust and Wearable Facial Expression Recognition via Deep-Learning-Assisted, Soft Epidermal Electronics
title_full_unstemmed Highly Robust and Wearable Facial Expression Recognition via Deep-Learning-Assisted, Soft Epidermal Electronics
title_sort highly robust and wearable facial expression recognition via deep-learning-assisted, soft epidermal electronics
publisher American Association for the Advancement of Science
series Research
issn 2639-5274
publishDate 2021-01-01
description The facial expressions are a mirror of the elusive emotion hidden in the mind, and thus, capturing expressions is a crucial way of merging the inward world and virtual world. However, typical facial expression recognition (FER) systems are restricted by environments where faces must be clearly seen for computer vision, or rigid devices that are not suitable for the time-dynamic, curvilinear faces. Here, we present a robust, highly wearable FER system that is based on deep-learning-assisted, soft epidermal electronics. The epidermal electronics that can fully conform on faces enable high-fidelity biosignal acquisition without hindering spontaneous facial expressions, releasing the constraint of movement, space, and light. The deep learning method can significantly enhance the recognition accuracy of facial expression types and intensities based on a small sample. The proposed wearable FER system is superior for wide applicability and high accuracy. The FER system is suitable for the individual and shows essential robustness to different light, occlusion, and various face poses. It is totally different from but complementary to the computer vision technology that is merely suitable for simultaneous FER of multiple individuals in a specific place. This wearable FER system is successfully applied to human-avatar emotion interaction and verbal communication disambiguation in a real-life environment, enabling promising human-computer interaction applications.
url http://dx.doi.org/10.34133/2021/9759601
work_keys_str_mv AT meiqizhuang highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics
AT langyin highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics
AT langyin highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics
AT youhuawang highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics
AT youhuawang highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics
AT yunzhaobai highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics
AT yunzhaobai highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics
AT jianzhan highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics
AT jianzhan highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics
AT chaohou highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics
AT chaohou highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics
AT litingyin highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics
AT litingyin highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics
AT zhangyuxu highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics
AT zhangyuxu highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics
AT xiaohuitan highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics
AT yonganhuang highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics
AT yonganhuang highlyrobustandwearablefacialexpressionrecognitionviadeeplearningassistedsoftepidermalelectronics
_version_ 1721282139280900096