Naturalistic Emotion Decoding From Facial Action Sets

• Main Authors: Sylwia Hyniewska, Wataru Sato, Susanne Kaiser, Catherine Pelachaud
• Format: Article
• Language: English
• Published: Frontiers Media S.A. 2019-01-01
• Series: Frontiers in Psychology
• Subjects: emotional facial expression; spontaneous expressions; naturalistic; cognitive appraisal; nonverbal behavior
• Online Access: https://www.frontiersin.org/article/10.3389/fpsyg.2018.02678/full

Researchers have theoretically proposed that humans decode other individuals' emotions or elementary cognitive appraisals from particular sets of facial action units (AUs). However, only a few empirical studies have systematically tested the relationships between the decoding of emotions/appraisals and sets of AUs, and the results are mixed. Furthermore, the previous studies relied on facial expressions of actors and no study used spontaneous and dynamic facial expressions in naturalistic settings. We investigated this issue using video recordings of facial expressions filmed unobtrusively in a real-life emotional situation, specifically loss of luggage at an airport. The AUs observed in the videos were annotated using the Facial Action Coding System. Male participants (n = 98) were asked to decode emotions (e.g., anger) and appraisals (e.g., suddenness) from facial expressions. We explored the relationships between the emotion/appraisal decoding and AUs using stepwise multiple regression analyses. The results revealed that all the rated emotions and appraisals were associated with sets of AUs. The profiles of regression equations showed AUs both consistent and inconsistent with those in theoretical proposals. The results suggest that (1) the decoding of emotions and appraisals in facial expressions is implemented by the perception of set of AUs, and (2) the profiles of such AU sets could be different from previous theories.