Enhanced Robust Univariate Classification Methods for Solving Outliers and Overfitting Problems

Abstract

The robustness of some classical univariate classifiers is hampered if the data are contaminated. Overfitting is another hiccup when the data sets are uncontaminated with a considerable sample size. The performance of the classification models can be easily biased by the outliers’ problems, of which the constructed model tends to be overfitted. Previous studies often used the Bayes Classifier (BC) and the Predictive Classifier (PC) to address two groups of univariate classification problems. Unfortunately for substantial large sample sizes and uncontaminated data, the BC method overfits when the Optimal Probability of Exact Classification (OPEC) is used as an evaluation benchmark. Meanwhile, for small sample sizes, the BC and PC methods are extremely susceptible to outliers. To overcome these two problems, we proposed two methods: the Smart Univariate Classifier (SUC) and the hybrid classifier. The latter is a combination of the SUC and the BC methods, known as the Smart Univariate Bayes Classifier (SUBC). The performance of the new classification methods was evaluated and compared with the conventional BC and PC methods using the OPEC as a benchmark value. To validate the performance of these classification methods, the Probability of Exact Classification (PEC) was compared with the OPEC value. The results showed that the proposed methods outperformed the conventional BC and PC methods based on the real data sets applied. Numerical results also revealed that the SUC method could solve the overfitting problem. The results further indicated that the two proposed methods were robust against outliers. Therefore, these new methods are helpful when practitioners are confronted with overfitting and data contamination problems.

Keywords

• Bayes
• Predictive
• Outliers
• Overfitting
• Classification