An Improved Binary Quadratic Discriminant Analysis Classifier by Using Robust Regularization

Abstract

In many real classification problems where a limited number of training samples is available, the linear classifiers based on discriminant analysis are unable to deliver accurate results. Moreover, the testing and/or the training data can be erroneous due to noise contamination which further degrades their performance. Regularization techniques become imperative to deal with these problems. However, the existing regularization techniques, to some extent, mainly focus on data scarcity issues but completely ignore the noisy nature of the testing and/or the training data. We propose a novel regularized quadratic discriminant analysis (R-QDA) classifier which addresses both issues simultaneously. The procedure involves a reformulation of the discriminant function of the conventional QDA classifier into least square problems and then solving them by using regularized least squares (Reg-LS) based on $\ell _{2}$ -norm. In contrast to existing R-QDA techniques, the proposed R-QDA classifier employs two regularization parameters pertaining to each class, which can be independently selected by various robust techniques. Numerical results demonstrate the effectiveness of the proposed method over classical R-QDA methods, especially in high noise regimes.

Keywords

• Discriminant analysis
• LDA
• QDA
• regularization
• data classification