Dynamic Analysis of Automatic Emotion Recognition Using Generalized Additive Mixed Models

Damien Dupre, Adam Booth, Andrew Bolster, Gawain Morrison, Gary McKeown

Research output: Chapter in Book/Report/Conference proceedingConference contribution

1 Citation (Scopus)
619 Downloads (Pure)

Abstract

The increasing ease of access to large participant populations via online recruitment platforms offers considerable promise to study aspects of emotion and participants’ relationship to brands, artistic media and advertising content. Large participant samples can be quickly and easily gathered, and their facial expressions can be assessed using video captured by webcams that are ubiquitous in modern computing environments. However, a long-standing issue is the degree of reliability of the data provided by automatic facial expression recognition systems. We evaluated the facial expressions of 836 participants triggered by an online sad video. Thanks to the use of this sizeable sample, this experiment has a high statistical power and showed, as expected, that Sadness expressions are significantly more frequently recognized compared to other facial expressions (i.e. Happiness, Surprise and Disgust). However, by using Generalized Additive Mixed Models (GAMM) to analyse the time-series we show that dynamic statistical analyses can avoid biased interpretations. Technical sensitivity, posture interpretation, emotion idiosyncrasy, intensity of spontaneous expressions and social context of the emotion elicitation are open questions that must be answered to perform reliable online facial expression recognition.
Original languageEnglish
Title of host publicationSymposium on Computational Modelling of Emotion: Theory and Applications
Pages158-163
Number of pages6
Publication statusPublished - 18 Apr 2017

Fingerprint

Dynamic analysis
Marketing
Time series
Experiments

Cite this

Dupre, D., Booth, A., Bolster, A., Morrison, G., & McKeown, G. (2017). Dynamic Analysis of Automatic Emotion Recognition Using Generalized Additive Mixed Models. In Symposium on Computational Modelling of Emotion: Theory and Applications (pp. 158-163)
Dupre, Damien ; Booth, Adam ; Bolster, Andrew ; Morrison, Gawain ; McKeown, Gary. / Dynamic Analysis of Automatic Emotion Recognition Using Generalized Additive Mixed Models. Symposium on Computational Modelling of Emotion: Theory and Applications. 2017. pp. 158-163
@inproceedings{d38e1948de174d848f2d391985913d0a,
title = "Dynamic Analysis of Automatic Emotion Recognition Using Generalized Additive Mixed Models",
abstract = "The increasing ease of access to large participant populations via online recruitment platforms offers considerable promise to study aspects of emotion and participants’ relationship to brands, artistic media and advertising content. Large participant samples can be quickly and easily gathered, and their facial expressions can be assessed using video captured by webcams that are ubiquitous in modern computing environments. However, a long-standing issue is the degree of reliability of the data provided by automatic facial expression recognition systems. We evaluated the facial expressions of 836 participants triggered by an online sad video. Thanks to the use of this sizeable sample, this experiment has a high statistical power and showed, as expected, that Sadness expressions are significantly more frequently recognized compared to other facial expressions (i.e. Happiness, Surprise and Disgust). However, by using Generalized Additive Mixed Models (GAMM) to analyse the time-series we show that dynamic statistical analyses can avoid biased interpretations. Technical sensitivity, posture interpretation, emotion idiosyncrasy, intensity of spontaneous expressions and social context of the emotion elicitation are open questions that must be answered to perform reliable online facial expression recognition.",
author = "Damien Dupre and Adam Booth and Andrew Bolster and Gawain Morrison and Gary McKeown",
year = "2017",
month = "4",
day = "18",
language = "English",
pages = "158--163",
booktitle = "Symposium on Computational Modelling of Emotion: Theory and Applications",

}

Dupre, D, Booth, A, Bolster, A, Morrison, G & McKeown, G 2017, Dynamic Analysis of Automatic Emotion Recognition Using Generalized Additive Mixed Models. in Symposium on Computational Modelling of Emotion: Theory and Applications. pp. 158-163.

Dynamic Analysis of Automatic Emotion Recognition Using Generalized Additive Mixed Models. / Dupre, Damien; Booth, Adam; Bolster, Andrew; Morrison, Gawain; McKeown, Gary.

Symposium on Computational Modelling of Emotion: Theory and Applications. 2017. p. 158-163.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Dynamic Analysis of Automatic Emotion Recognition Using Generalized Additive Mixed Models

AU - Dupre, Damien

AU - Booth, Adam

AU - Bolster, Andrew

AU - Morrison, Gawain

AU - McKeown, Gary

PY - 2017/4/18

Y1 - 2017/4/18

N2 - The increasing ease of access to large participant populations via online recruitment platforms offers considerable promise to study aspects of emotion and participants’ relationship to brands, artistic media and advertising content. Large participant samples can be quickly and easily gathered, and their facial expressions can be assessed using video captured by webcams that are ubiquitous in modern computing environments. However, a long-standing issue is the degree of reliability of the data provided by automatic facial expression recognition systems. We evaluated the facial expressions of 836 participants triggered by an online sad video. Thanks to the use of this sizeable sample, this experiment has a high statistical power and showed, as expected, that Sadness expressions are significantly more frequently recognized compared to other facial expressions (i.e. Happiness, Surprise and Disgust). However, by using Generalized Additive Mixed Models (GAMM) to analyse the time-series we show that dynamic statistical analyses can avoid biased interpretations. Technical sensitivity, posture interpretation, emotion idiosyncrasy, intensity of spontaneous expressions and social context of the emotion elicitation are open questions that must be answered to perform reliable online facial expression recognition.

AB - The increasing ease of access to large participant populations via online recruitment platforms offers considerable promise to study aspects of emotion and participants’ relationship to brands, artistic media and advertising content. Large participant samples can be quickly and easily gathered, and their facial expressions can be assessed using video captured by webcams that are ubiquitous in modern computing environments. However, a long-standing issue is the degree of reliability of the data provided by automatic facial expression recognition systems. We evaluated the facial expressions of 836 participants triggered by an online sad video. Thanks to the use of this sizeable sample, this experiment has a high statistical power and showed, as expected, that Sadness expressions are significantly more frequently recognized compared to other facial expressions (i.e. Happiness, Surprise and Disgust). However, by using Generalized Additive Mixed Models (GAMM) to analyse the time-series we show that dynamic statistical analyses can avoid biased interpretations. Technical sensitivity, posture interpretation, emotion idiosyncrasy, intensity of spontaneous expressions and social context of the emotion elicitation are open questions that must be answered to perform reliable online facial expression recognition.

M3 - Conference contribution

SP - 158

EP - 163

BT - Symposium on Computational Modelling of Emotion: Theory and Applications

ER -

Dupre D, Booth A, Bolster A, Morrison G, McKeown G. Dynamic Analysis of Automatic Emotion Recognition Using Generalized Additive Mixed Models. In Symposium on Computational Modelling of Emotion: Theory and Applications. 2017. p. 158-163