Personalized machine learning for facial expression analysis

• Main Author: Feffer, Michael A. (Michael Anthony)
• Other Authors: Ognjen (Oggi) Rudovic and Rosalind Picard.
• Format: Others
• Language: English
• Published: Massachusetts Institute of Technology 2018
• Subjects: Electrical Engineering and Computer Science.
• Online Access: http://hdl.handle.net/1721.1/119763

Thesis: M. Eng., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2018. === This electronic version was submitted by the student author. The certified thesis is available in the Institute Archives and Special Collections. === Cataloged from student-submitted PDF version of thesis. === Includes bibliographical references (pages 35-36). === For this MEng Thesis Project, I investigated the personalization of deep convolutional networks for facial expression analysis. While prior work focused on population-based ("one-size-fits-all") models for prediction of affective states (valence/arousal), I constructed personalized versions of these models to improve upon state-of-the-art general models through solving a domain adaptation problem. This was done by starting with pre-trained deep models for face analysis and fine-tuning the last layers to specific subjects or subpopulations. For prediction, a "mixture of experts" (MoE) solution was employed to select the proper outputs based on the given input. The research questions answered in this project are: (1) What are the effects of model personalization on the estimation of valence and arousal from faces? (2) What is the amount of (un)supervised data needed to reach a target performance? Models produced in this research provide the foundation of a novel tool for personalized real-time estimation of target metrics. === by Michael A. Feffer. === M. Eng.