Imbalance Robust Softmax for Deep Embedding Learning

Hao Zhu, Yang Yuan, Guosheng Hu, Xiang Wu, Neil Robertson

Research output: Chapter in Book/Report/Conference proceedingConference contribution

72 Downloads (Pure)


Deep embedding learning is expected to learn a metric space in which features have smaller maximal intra-class distance than minimal inter-class distance. In recent years, one research focus is to solve the open-set problem by discriminative deep embedding learning in the field of face recognition (FR) and person re-identification (re-ID). Apart from open-set problem, we find that imbalanced training data is another main factor causing the performance degradation of FR and re-ID, and data imbalance widely exists in the real applications. However, very little research explores why and how data imbalance influences the performance of FR and re-ID with softmax or its variants. In this work, we deeply investigate data imbalance in the perspective of neural network optimisation and feature distribution about softmax. We find one main reason of performance degradation caused by data imbalance is that the weights (from the penultimate fully-connected layer) are far from their class centers in feature space. Based on this investigation, we propose a unified framework, Imbalance-Robust Softmax (IR-Softmax), which can simultaneously solve the open-set problem and reduce the influence of data imbalance. IR-Softmax can generalise to any softmax and its variants (which are discriminative for open-set problem) by directly setting the weights as their class centers, naturally solving the data imbalance problem. In this work, we explicitly re-formulate two discriminative softmax (A-Softmax and AM-Softmax) under the framework of IR-Softmax. We conduct extensive experiments on FR databases (LFW, MegaFace) and re-ID database (Market-1501, Duke), and IR-Softmax outperforms many state-of-the-art methods.
Original languageEnglish
Title of host publication Asian Conference on Computer Vision 2020 30/11/2020
Subtitle of host publicationKyoto, Japan
Publication statusPublished - 26 Feb 2021
EventAsian Conference on Computer Vision 2020 - Kyoto, Japan
Duration: 30 Nov 2020 → …

Publication series

NameLecture Notes in Computer Science
ISSN (Print)0302-9743


ConferenceAsian Conference on Computer Vision 2020
Abbreviated titleACCV
Period30/11/2020 → …
Internet address


Dive into the research topics of 'Imbalance Robust Softmax for Deep Embedding Learning'. Together they form a unique fingerprint.

Cite this