Large-Dimensional Characterization of Robust Linear Discriminant Analysis

Nicolas Auguin, David Morales-Jimenez, Matthew R. Mckay

Research output: Contribution to journalArticlepeer-review

Abstract

In standard discriminant analysis, data are commonly assumed to follow a Gaussian distribution, a condition which is often violated in practice. In this work, to account for potential spurious or mislabeled observations in the training data, we consider a robust version of regularized linear discriminant analysis (LDA) classifiers. Essential to such robust version of LDA is the design of a robust discriminant rule which relies on a robust estimate of the covariance matrix of the training data. We propose to use a regularized version of M-estimators of covariance matrices belonging to Maronna's class of estimators. In the regime where both the number of variables and the number of training samples are large, building upon recent results from random matrix theory, we show that when the training data are free from outliers, each classifier within the class of proposed robust classifiers is asymptotically equivalent to traditional, non-robust classifiers. Rather surprisingly, this entails that the use of robust estimators does not degrade the performance of LDA, up to a transformation of the regularization parameter that we precisely characterize. We also demonstrate that the proposed robust classifiers lead to a better classification accuracy when the data are corrupted by outliers or random noise. Furthermore, through simulations on the popular MNIST data set and considering different classification tasks, we show that the worse the classification error of traditional methods is, the further gain is to be expected with the use of our proposed method.

Original languageEnglish
Pages (from-to)2625–2638
Number of pages14
JournalIEEE Transactions on Signal Processing
Volume69
DOIs
Publication statusPublished - 23 Apr 2021

Bibliographical note

Publisher Copyright:
IEEE

Copyright:
Copyright 2021 Elsevier B.V., All rights reserved.

Keywords

  • Analytical models
  • covariance matrices
  • Covariance matrices
  • Data models
  • Estimation
  • linear discriminant analysis
  • Robust estimation
  • Standards
  • Training
  • Training data

ASJC Scopus subject areas

  • Signal Processing
  • Electrical and Electronic Engineering

Fingerprint

Dive into the research topics of 'Large-Dimensional Characterization of Robust Linear Discriminant Analysis'. Together they form a unique fingerprint.

Cite this