Abstract
Facial expressions play an important role during communications, allowing information regarding the emotional state of an individual to be conveyed and inferred. Research suggests that automatic facial expression recognition is a promising avenue of enquiry in mental healthcare, as facial expressions can also reflect an individual's mental state. In order to develop user-friendly, low-cost and effective facial expression analysis systems for mental health care, this paper presents a novel deep convolution network based emotion analysis framework to support mental state detection and diagnosis. The proposed system is able to process facial images and interpret the temporal evolution of emotions through a new solution in which deep features are extracted from the Fully Connected Layer 6 of the AlexNet, with a standard Linear Discriminant Analysis Classifier exploited to obtain the final classification outcome. It is tested against 5 benchmarking databases, including JAFFE, KDEF,CK+, and databases with the images obtained ‘in the wild’ such as FER2013 and AffectNet. Compared with the other state-of-the-art methods, we observe that our method has overall higher accuracy of facial expression recognition. Additionally, when compared to the state-of-the-art deep learning algorithms such as Vgg16, GoogleNet, ResNet and AlexNet, the proposed method demonstrated better efficiency and has less device requirements. The experiments presented in this paper demonstrate that the proposed method outperforms the other methods in terms of accuracy and efficiency which suggests it could act as a smart, low-cost, user-friendly cognitive aid to detect, monitor, and diagnose the mental health of a patient through automatic facial expression analysis.
| Original language | English |
|---|---|
| Pages (from-to) | 212-227 |
| Number of pages | 16 |
| Journal | Neurocomputing |
| Volume | 388 |
| Early online date | 16 Jan 2020 |
| DOIs | |
| Publication status | Published - 07 May 2020 |
| Externally published | Yes |
Bibliographical note
Funding Information:The research work is funded by Strathclyde's Strategic Technology Partnership (STP) Programme with CAPITA (2016?2019). We thank Dr. Neil Mackin and Miss Angela Anderson for their support. The contents in the paper are those of the authors alone and don't stand for the views of CAPITA plc. Huiyu Zhou was partly funded by UK EPSRC under Grant EP/N011074/1, and Royal Society in Newton Advanced Fellowship under Grant NA160342. Winifred Ijomah is supported under EPSRC (Grant no. EP/N018427/1). The authors thank Shanghai Mental Health Center for their help and support. We also thank Dr. Fei Gao from Beihang University, China for his kind support and comment. This article does not contain any studies with human participants or animals performed by any of the authors without the proper ethical approval.
Funding Information:
The research work is funded by Strathclyde's Strategic Technology Partnership (STP) Programme with CAPITA (2016–2019). We thank Dr. Neil Mackin and Miss Angela Anderson for their support. The contents in the paper are those of the authors alone and don't stand for the views of CAPITA plc. Huiyu Zhou was partly funded by UK EPSRC under Grant EP/N011074/1 , and Royal Society in Newton Advanced Fellowship under Grant NA160342 . The authors thank Shanghai Mental Health Center for their help and support. We also thank Dr. Fei Gao from Beihang University, China for his kind support and comment.
Publisher Copyright:
© 2020
Copyright:
Copyright 2020 Elsevier B.V., All rights reserved.
Keywords
- Deep convolution network
- Emotion analysis
- Facial expression recognition
- Mental health care
ASJC Scopus subject areas
- Computer Science Applications
- Cognitive Neuroscience
- Artificial Intelligence
