Computational Analysis of Deep Visual Data for Quantifying Facial Expression Production

The computational analysis of facial expressions is an emerging research topic that could overcome the limitations of human perception and get quick and objective outcomes in the assessment of neurodevelopmental disorders (e.g., Autism Spectrum Disorders, ASD). Unfortunately, there have been only a...

Full description

Bibliographic Details
Main Authors: Marco Leo, Pierluigi Carcagnì, Cosimo Distante, Pier Luigi Mazzeo, Paolo Spagnolo, Annalisa Levante, Serena Petrocchi, Flavia Lecciso
Format: Article
Language:English
Published: MDPI AG 2019-10-01
Series:Applied Sciences
Subjects:
Online Access:https://www.mdpi.com/2076-3417/9/21/4542
id doaj-b62ef27283ef4128a4918c937d03522c
record_format Article
spelling doaj-b62ef27283ef4128a4918c937d03522c2020-11-25T01:56:34ZengMDPI AGApplied Sciences2076-34172019-10-01921454210.3390/app9214542app9214542Computational Analysis of Deep Visual Data for Quantifying Facial Expression ProductionMarco Leo0Pierluigi Carcagnì1Cosimo Distante2Pier Luigi Mazzeo3Paolo Spagnolo4Annalisa Levante5Serena Petrocchi6Flavia Lecciso7Institute of Applied Sciences and Intelligent Systems, National Research Council, 73100 Lecce, ItalyInstitute of Applied Sciences and Intelligent Systems, National Research Council, 73100 Lecce, ItalyInstitute of Applied Sciences and Intelligent Systems, National Research Council, 73100 Lecce, ItalyInstitute of Applied Sciences and Intelligent Systems, National Research Council, 73100 Lecce, ItalyInstitute of Applied Sciences and Intelligent Systems, National Research Council, 73100 Lecce, ItalyDepartment of History, Society and Human Studies, Università del Salento, 73100 Lecce, ItalyLab of Applied Psychology and Intervention, Università del Salento, 73100 Lecce Le, ItalyDepartment of History, Society and Human Studies, Università del Salento, 73100 Lecce, ItalyThe computational analysis of facial expressions is an emerging research topic that could overcome the limitations of human perception and get quick and objective outcomes in the assessment of neurodevelopmental disorders (e.g., Autism Spectrum Disorders, ASD). Unfortunately, there have been only a few attempts to quantify facial expression production and most of the scientific literature aims at the easier task of recognizing if either a facial expression is present or not. Some attempts to face this challenging task exist but they do not provide a comprehensive study based on the comparison between human and automatic outcomes in quantifying children’s ability to produce basic emotions. Furthermore, these works do not exploit the latest solutions in computer vision and machine learning. Finally, they generally focus only on a homogeneous (in terms of cognitive capabilities) group of individuals. To fill this gap, in this paper some advanced computer vision and machine learning strategies are integrated into a framework aimed to computationally analyze how both ASD and typically developing children produce facial expressions. The framework locates and tracks a number of landmarks (virtual electromyography sensors) with the aim of monitoring facial muscle movements involved in facial expression production. The output of these virtual sensors is then fused to model the individual ability to produce facial expressions. Gathered computational outcomes have been correlated with the evaluation provided by psychologists and evidence has been given that shows how the proposed framework could be effectively exploited to deeply analyze the emotional competence of ASD children to produce facial expressions.https://www.mdpi.com/2076-3417/9/21/4542assistive technologyautismfacial expressionscomputer vision
collection DOAJ
language English
format Article
sources DOAJ
author Marco Leo
Pierluigi Carcagnì
Cosimo Distante
Pier Luigi Mazzeo
Paolo Spagnolo
Annalisa Levante
Serena Petrocchi
Flavia Lecciso
spellingShingle Marco Leo
Pierluigi Carcagnì
Cosimo Distante
Pier Luigi Mazzeo
Paolo Spagnolo
Annalisa Levante
Serena Petrocchi
Flavia Lecciso
Computational Analysis of Deep Visual Data for Quantifying Facial Expression Production
Applied Sciences
assistive technology
autism
facial expressions
computer vision
author_facet Marco Leo
Pierluigi Carcagnì
Cosimo Distante
Pier Luigi Mazzeo
Paolo Spagnolo
Annalisa Levante
Serena Petrocchi
Flavia Lecciso
author_sort Marco Leo
title Computational Analysis of Deep Visual Data for Quantifying Facial Expression Production
title_short Computational Analysis of Deep Visual Data for Quantifying Facial Expression Production
title_full Computational Analysis of Deep Visual Data for Quantifying Facial Expression Production
title_fullStr Computational Analysis of Deep Visual Data for Quantifying Facial Expression Production
title_full_unstemmed Computational Analysis of Deep Visual Data for Quantifying Facial Expression Production
title_sort computational analysis of deep visual data for quantifying facial expression production
publisher MDPI AG
series Applied Sciences
issn 2076-3417
publishDate 2019-10-01
description The computational analysis of facial expressions is an emerging research topic that could overcome the limitations of human perception and get quick and objective outcomes in the assessment of neurodevelopmental disorders (e.g., Autism Spectrum Disorders, ASD). Unfortunately, there have been only a few attempts to quantify facial expression production and most of the scientific literature aims at the easier task of recognizing if either a facial expression is present or not. Some attempts to face this challenging task exist but they do not provide a comprehensive study based on the comparison between human and automatic outcomes in quantifying children’s ability to produce basic emotions. Furthermore, these works do not exploit the latest solutions in computer vision and machine learning. Finally, they generally focus only on a homogeneous (in terms of cognitive capabilities) group of individuals. To fill this gap, in this paper some advanced computer vision and machine learning strategies are integrated into a framework aimed to computationally analyze how both ASD and typically developing children produce facial expressions. The framework locates and tracks a number of landmarks (virtual electromyography sensors) with the aim of monitoring facial muscle movements involved in facial expression production. The output of these virtual sensors is then fused to model the individual ability to produce facial expressions. Gathered computational outcomes have been correlated with the evaluation provided by psychologists and evidence has been given that shows how the proposed framework could be effectively exploited to deeply analyze the emotional competence of ASD children to produce facial expressions.
topic assistive technology
autism
facial expressions
computer vision
url https://www.mdpi.com/2076-3417/9/21/4542
work_keys_str_mv AT marcoleo computationalanalysisofdeepvisualdataforquantifyingfacialexpressionproduction
AT pierluigicarcagni computationalanalysisofdeepvisualdataforquantifyingfacialexpressionproduction
AT cosimodistante computationalanalysisofdeepvisualdataforquantifyingfacialexpressionproduction
AT pierluigimazzeo computationalanalysisofdeepvisualdataforquantifyingfacialexpressionproduction
AT paolospagnolo computationalanalysisofdeepvisualdataforquantifyingfacialexpressionproduction
AT annalisalevante computationalanalysisofdeepvisualdataforquantifyingfacialexpressionproduction
AT serenapetrocchi computationalanalysisofdeepvisualdataforquantifyingfacialexpressionproduction
AT flavialecciso computationalanalysisofdeepvisualdataforquantifyingfacialexpressionproduction
_version_ 1724979293039099904