Combining deep neural network with traditional classifier to recognize facial expressions

Zixiang Fei; Erfu Yang; David Li; Stephen Butler; Winifred Ijomah; Huiyu Zhou

doi:10.23919/IConAC.2019.8895084

Combining deep neural network with traditional classifier to recognize facial expressions

Zixiang Fei, Erfu Yang^*, David Li, Stephen Butler, Winifred Ijomah, Huiyu Zhou

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

11 Citations (Scopus)

Abstract

Facial expressions are important in people's daily communications. Recognising facial expressions also has many important applications in the areas such as healthcare and e-learning. Existing facial expression recognition systems have problems such as background interference. Furthermore, systems using traditional approaches like SVM (Support Vector Machine) have weakness in dealing with unseen images. Systems using deep neural network have problems such as requirement for GPU, longer training time and requirement for large memory. To overcome the shortcomings of pure deep neural network and traditional facial recognition approaches, this paper presents a new facial expression recognition approach which has image pre-processing techniques to remove unnecessary background information and combines deep neural network ResNet50 and a traditional classifier-the multiclass model for Support Vector Machine to recognise facial expressions. The proposed approach has better recognition accuracy than traditional approaches like Support Vector Machine and doesn't need GPU. We have compared 3 proposed frameworks with a traditional SVM approach against the Karolinska Directed Emotional Faces (KDEF) Database, the Japanese Female Facial Expression (JAFFE) Database and the extended Cohn-Kanade dataset (CK+), respectively. The experiment results show that the features extracted from the layer 49Relu have the best performance for these three datasets.

Original language	English
Title of host publication	ICAC 2019 - 2019 25th IEEE International Conference on Automation and Computing
Editors	Hui Yu
Publisher	Institute of Electrical and Electronics Engineers Inc.
Number of pages	6
ISBN (Electronic)	9781861376664
DOIs	https://doi.org/10.23919/IConAC.2019.8895084
Publication status	Published - 11 Nov 2019
Externally published	Yes
Event	25th IEEE International Conference on Automation and Computing, ICAC 2019 - Lancaster, United Kingdom Duration: 05 Sept 2019 → 07 Sept 2019

Publication series

Name	ICAC 2019 - 2019 25th IEEE International Conference on Automation and Computing

Conference

Conference	25th IEEE International Conference on Automation and Computing, ICAC 2019
Country/Territory	United Kingdom
City	Lancaster
Period	05/09/2019 → 07/09/2019

Bibliographical note

Funding Information:
ACKNOWLEDGEMENTS This research is funded by Strathclyde’s Strategic Technology Partnership (STP) Programme with CAPITA (2016-2019). The authors thank Dr Neil Mackin (CAPITA mentor) and Miss Angela Anderson (the STP’s coordinator) for their support. The contents including any opinions and conclusions made in this paper are those of the authors alone. They do not necessarily represent the views of CAPITA plc. Huiyu Zhou was partly funded by UK EPSRC under Grant EP/N011074/1, and Royal Society-Newton Advanced Fellowship under Grant NA160342. The authors thank Shanghai Mental Health Center for its valuable advice. The discussion and support from Dr Fei Gao from Beihang University, China are also appreciated and acknowledgements.

Publisher Copyright:
© 2019 Chinese Automation and Computing Society in the UK-CACSUK.

Copyright:
Copyright 2020 Elsevier B.V., All rights reserved.

Keywords

Deep Convolution Network
Facial Expression Recognition
Support Vector Machine

ASJC Scopus subject areas

Artificial Intelligence
Computer Networks and Communications
Computer Science Applications
Control and Optimization

Access to Document

10.23919/IConAC.2019.8895084Licence: Unspecified

Cite this

Fei, Z., Yang, E., Li, D., Butler, S., Ijomah, W., & Zhou, H. (2019). Combining deep neural network with traditional classifier to recognize facial expressions. In H. Yu (Ed.), ICAC 2019 - 2019 25th IEEE International Conference on Automation and Computing (ICAC 2019 - 2019 25th IEEE International Conference on Automation and Computing). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.23919/IConAC.2019.8895084

Fei, Zixiang ; Yang, Erfu ; Li, David et al. / Combining deep neural network with traditional classifier to recognize facial expressions. ICAC 2019 - 2019 25th IEEE International Conference on Automation and Computing. editor / Hui Yu. Institute of Electrical and Electronics Engineers Inc., 2019. (ICAC 2019 - 2019 25th IEEE International Conference on Automation and Computing).

@inproceedings{60cd78e59a8e4b9aa28f983b87fe920a,

title = "Combining deep neural network with traditional classifier to recognize facial expressions",

abstract = "Facial expressions are important in people's daily communications. Recognising facial expressions also has many important applications in the areas such as healthcare and e-learning. Existing facial expression recognition systems have problems such as background interference. Furthermore, systems using traditional approaches like SVM (Support Vector Machine) have weakness in dealing with unseen images. Systems using deep neural network have problems such as requirement for GPU, longer training time and requirement for large memory. To overcome the shortcomings of pure deep neural network and traditional facial recognition approaches, this paper presents a new facial expression recognition approach which has image pre-processing techniques to remove unnecessary background information and combines deep neural network ResNet50 and a traditional classifier-the multiclass model for Support Vector Machine to recognise facial expressions. The proposed approach has better recognition accuracy than traditional approaches like Support Vector Machine and doesn't need GPU. We have compared 3 proposed frameworks with a traditional SVM approach against the Karolinska Directed Emotional Faces (KDEF) Database, the Japanese Female Facial Expression (JAFFE) Database and the extended Cohn-Kanade dataset (CK+), respectively. The experiment results show that the features extracted from the layer 49Relu have the best performance for these three datasets.",

keywords = "Deep Convolution Network, Facial Expression Recognition, Support Vector Machine",

author = "Zixiang Fei and Erfu Yang and David Li and Stephen Butler and Winifred Ijomah and Huiyu Zhou",

note = "Funding Information: ACKNOWLEDGEMENTS This research is funded by Strathclyde{\textquoteright}s Strategic Technology Partnership (STP) Programme with CAPITA (2016-2019). The authors thank Dr Neil Mackin (CAPITA mentor) and Miss Angela Anderson (the STP{\textquoteright}s coordinator) for their support. The contents including any opinions and conclusions made in this paper are those of the authors alone. They do not necessarily represent the views of CAPITA plc. Huiyu Zhou was partly funded by UK EPSRC under Grant EP/N011074/1, and Royal Society-Newton Advanced Fellowship under Grant NA160342. The authors thank Shanghai Mental Health Center for its valuable advice. The discussion and support from Dr Fei Gao from Beihang University, China are also appreciated and acknowledgements. Publisher Copyright: {\textcopyright} 2019 Chinese Automation and Computing Society in the UK-CACSUK. Copyright: Copyright 2020 Elsevier B.V., All rights reserved.; 25th IEEE International Conference on Automation and Computing, ICAC 2019 ; Conference date: 05-09-2019 Through 07-09-2019",

year = "2019",

month = nov,

day = "11",

doi = "10.23919/IConAC.2019.8895084",

language = "English",

series = "ICAC 2019 - 2019 25th IEEE International Conference on Automation and Computing",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

editor = "Hui Yu",

booktitle = "ICAC 2019 - 2019 25th IEEE International Conference on Automation and Computing",

address = "United States",

}

Fei, Z, Yang, E, Li, D, Butler, S, Ijomah, W & Zhou, H 2019, Combining deep neural network with traditional classifier to recognize facial expressions. in H Yu (ed.), ICAC 2019 - 2019 25th IEEE International Conference on Automation and Computing. ICAC 2019 - 2019 25th IEEE International Conference on Automation and Computing, Institute of Electrical and Electronics Engineers Inc., 25th IEEE International Conference on Automation and Computing, ICAC 2019, Lancaster, United Kingdom, 05/09/2019. https://doi.org/10.23919/IConAC.2019.8895084

Combining deep neural network with traditional classifier to recognize facial expressions. / Fei, Zixiang; Yang, Erfu; Li, David et al.
ICAC 2019 - 2019 25th IEEE International Conference on Automation and Computing. ed. / Hui Yu. Institute of Electrical and Electronics Engineers Inc., 2019. (ICAC 2019 - 2019 25th IEEE International Conference on Automation and Computing).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

TY - GEN

T1 - Combining deep neural network with traditional classifier to recognize facial expressions

AU - Fei, Zixiang

AU - Yang, Erfu

AU - Li, David

AU - Butler, Stephen

AU - Ijomah, Winifred

AU - Zhou, Huiyu

N1 - Funding Information: ACKNOWLEDGEMENTS This research is funded by Strathclyde’s Strategic Technology Partnership (STP) Programme with CAPITA (2016-2019). The authors thank Dr Neil Mackin (CAPITA mentor) and Miss Angela Anderson (the STP’s coordinator) for their support. The contents including any opinions and conclusions made in this paper are those of the authors alone. They do not necessarily represent the views of CAPITA plc. Huiyu Zhou was partly funded by UK EPSRC under Grant EP/N011074/1, and Royal Society-Newton Advanced Fellowship under Grant NA160342. The authors thank Shanghai Mental Health Center for its valuable advice. The discussion and support from Dr Fei Gao from Beihang University, China are also appreciated and acknowledgements. Publisher Copyright: © 2019 Chinese Automation and Computing Society in the UK-CACSUK. Copyright: Copyright 2020 Elsevier B.V., All rights reserved.

PY - 2019/11/11

Y1 - 2019/11/11

N2 - Facial expressions are important in people's daily communications. Recognising facial expressions also has many important applications in the areas such as healthcare and e-learning. Existing facial expression recognition systems have problems such as background interference. Furthermore, systems using traditional approaches like SVM (Support Vector Machine) have weakness in dealing with unseen images. Systems using deep neural network have problems such as requirement for GPU, longer training time and requirement for large memory. To overcome the shortcomings of pure deep neural network and traditional facial recognition approaches, this paper presents a new facial expression recognition approach which has image pre-processing techniques to remove unnecessary background information and combines deep neural network ResNet50 and a traditional classifier-the multiclass model for Support Vector Machine to recognise facial expressions. The proposed approach has better recognition accuracy than traditional approaches like Support Vector Machine and doesn't need GPU. We have compared 3 proposed frameworks with a traditional SVM approach against the Karolinska Directed Emotional Faces (KDEF) Database, the Japanese Female Facial Expression (JAFFE) Database and the extended Cohn-Kanade dataset (CK+), respectively. The experiment results show that the features extracted from the layer 49Relu have the best performance for these three datasets.

AB - Facial expressions are important in people's daily communications. Recognising facial expressions also has many important applications in the areas such as healthcare and e-learning. Existing facial expression recognition systems have problems such as background interference. Furthermore, systems using traditional approaches like SVM (Support Vector Machine) have weakness in dealing with unseen images. Systems using deep neural network have problems such as requirement for GPU, longer training time and requirement for large memory. To overcome the shortcomings of pure deep neural network and traditional facial recognition approaches, this paper presents a new facial expression recognition approach which has image pre-processing techniques to remove unnecessary background information and combines deep neural network ResNet50 and a traditional classifier-the multiclass model for Support Vector Machine to recognise facial expressions. The proposed approach has better recognition accuracy than traditional approaches like Support Vector Machine and doesn't need GPU. We have compared 3 proposed frameworks with a traditional SVM approach against the Karolinska Directed Emotional Faces (KDEF) Database, the Japanese Female Facial Expression (JAFFE) Database and the extended Cohn-Kanade dataset (CK+), respectively. The experiment results show that the features extracted from the layer 49Relu have the best performance for these three datasets.

KW - Deep Convolution Network

KW - Facial Expression Recognition

KW - Support Vector Machine

UR - https://www.scopus.com/pages/publications/85075795064

U2 - 10.23919/IConAC.2019.8895084

DO - 10.23919/IConAC.2019.8895084

M3 - Conference contribution

AN - SCOPUS:85075795064

T3 - ICAC 2019 - 2019 25th IEEE International Conference on Automation and Computing

BT - ICAC 2019 - 2019 25th IEEE International Conference on Automation and Computing

A2 - Yu, Hui

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 25th IEEE International Conference on Automation and Computing, ICAC 2019

Y2 - 5 September 2019 through 7 September 2019

ER -

Fei Z, Yang E, Li D, Butler S, Ijomah W, Zhou H. Combining deep neural network with traditional classifier to recognize facial expressions. In Yu H, editor, ICAC 2019 - 2019 25th IEEE International Conference on Automation and Computing. Institute of Electrical and Electronics Engineers Inc. 2019. (ICAC 2019 - 2019 25th IEEE International Conference on Automation and Computing). doi: 10.23919/IConAC.2019.8895084

Combining deep neural network with traditional classifier to recognize facial expressions

Abstract

Publication series

Conference

Bibliographical note

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this