Attribute-Enhanced Face Recognition with Neural Tensor Fusion Networks

Guosheng Hu, Yang Hua, Yangyuan Yuan, Zhihong Zhang, Sankha Subhra Mukherjee, Timothy Hospedales, Neil Robertson, Yongxin Yang

Research output: Chapter in Book/Report/Conference proceedingConference contribution

37 Citations (Scopus)
400 Downloads (Pure)

Abstract

Deep learning has achieved great success in face recognition, however deep-learned features still have limited invariance to strong intra-personal variations such as large
pose changes. It is observed that some facial attributes (e.g. eyebrow thickness, gender) are robust to such variations. We present the first work to systematically explore how the fusion of face recognition features (FRF) and facial attribute features (FAF) can enhance face recognition performance in various challenging scenarios. Despite the promise of FAF, we find that in practice existing fusion methods fail to leverage FAF to boost face recognition performance in some challenging scenarios. Thus, we develop a powerful tensor-based framework which formulates feature fusion as a tensor optimisation problem. It is nontrivial to directly optimise this tensor due to the large number of parameters to optimise. To solve this problem, we establish a theoretical equivalence between low-rank tensor optimisation and a two-stream gated neural network. This equivalence allows tractable learning using standard neural network optimisation tools, leading to accurate and stable optimisation. Experimental results show the fused feature works better than individual features, thus proving for the first time that facial attributes aid face recognition. We achieve state-of-the-art performance on three popular databases: MultiPIE (cross pose, lighting and expression), CASIA NIR-VIS2.0 (cross-modality environment) and LFW (uncontrolled environment).
Original languageEnglish
Title of host publicationInternational Conference on Computer Vision (ICCV) 2017: Proceedings
Publisher IEEE
Pages3764-3773
Number of pages10
ISBN (Electronic)978-1-5386-1032-9
ISBN (Print)978-1-5386-1033-6
Publication statusPublished - 25 Dec 2017
EventInternational Conference on Computer Vision 2017 - Venice, Italy
Duration: 22 Oct 201729 Oct 2017

Publication series

NameIEEE International Conference on Computer Vision (ICCV)
PublisherIEEE
ISSN (Print)2380-7504

Conference

ConferenceInternational Conference on Computer Vision 2017
Country/TerritoryItaly
CityVenice
Period22/10/201729/10/2017

Fingerprint

Dive into the research topics of 'Attribute-Enhanced Face Recognition with Neural Tensor Fusion Networks'. Together they form a unique fingerprint.

Cite this