Efficient Feature Selection Algorithm Based on Particle Swarm Optimization With Learning Memory

Abstract

Feature selection is an important pre-processing step in machine learning and data mining tasks, which improves the performance of the learning models by removing redundant and irrelevant features. Many feature selection algorithms have been widely studied, including greedy and random search approaches, to find a subset of the most important features for fulfilling a particular task (i.e., classification and regression). As a powerful swarm-based meta-heuristic method, particle swarm optimization (PSO) is reported to be suitable for optimization problems with continuous search space. However, the traditional PSO has rarely been applied to feature selection as a discrete space search problem. In this paper, a novel feature selection algorithm based on PSO with learning memory (PSO-LM) is proposed. The goal of the learning memory strategy is designed to inherit much more useful knowledge from those individuals who have higher fitness and offer faster progress, and the genetic operation is used to balance the local exploitation and the global exploration of the algorithm. Moreover, the k-nearest neighbor method is used as a classifier to evaluate the classification accuracy of a particle. The proposed method has been evaluated on some international standard data sets, and the results demonstrated its superiority compared with those wrapper-based feature selection methods.

Keywords

• Combinatorial optimization
• feature selection
• global optimization
• learning memory
• particle swarm optimization