Improving Classification Performance with Statistically Weighted Dimensions and Dimensionality Reduction

Abstract

In image classification, various techniques have been developed to enhance the performance of principal component analysis (PCA) dimension reduction techniques with guiding weighting features to remove redundant and irrelevant features. This study proposes the statistically weighted dimension technique based on three distribution-related class behaviors; collection-class, inter-class, and intra-class to enhance the feature-extraction ability before using PCA for feature selection. The data from the statistics-weighted dimension spaces is utilized to reduce dimensionality by reducing the large index data into smaller index data using PCA. The new principal component from the weighted training part by an unlabeled dataset is constructed and then the image is classified efficiently. Additionally, the weighting direction investigates the pros and cons of promoting and demoting to determine the worst or best option utilizing the exponents of three proposed weighted scheme. The experiment is conducted using three datasets, MNIST, E-MNIST, and F-MNIST, along with three image classification algorithms, logistic Regression, KNN, and SVM (RBF). The results clearly demonstrate that the statistically weighted dimension feature can improve the conventional classification accuracy in lower dimensions with an appropriate combination of weighting nearly 3% for the best solution on dimensionality reduction by more than 50%.

Keywords

• statistically weighted dimension
• feature weighting
• dimension reduction
• class character
• collection-class
• inter-class