Detection and visualization of non-linear structures in large datasets using Exploratory Projection Pursuit Laboratory (EPP-Lab) software

Abstract

This article consists of using biologically inspired algorithms in order to detect potentially interesting structures in large and multidimensional data sets. Data exploration and the detection of interesting structures are based on the use of Projection Pursuit that involves the definition and the optimization of an index associated with each direction or projection. The optimization of a projection index should provide a set of multiple optima that is expected to correspond to interesting graphical representations in low dimensional space. The implementation of the bio-inspired algorithms along with the projection pursuit develops a new software called EPP-Lab. Projection pursuit is widely used in different scientific domains (biology, pharmacy, bioinformatics, biometry, etc) but not widely present in the well-known softwares. EPP-Lab is dedicated to recognize and visualize clusters and outlying observations on one dimension from high dimensional and multivariate data sets. It includes different statistical techniques for results analysis. It provides several features and gives the user the option to adjust the parameters of the selected bio-inspired methods or to use defaults values. EPP-Lab is a unique software for detection, visualization and analysis of non-linear structures. The performance of this tool has been validated by testing different real and simulated data sets.

Keywords

• Exploratory Projection Pursuit
• Genetic Algorithm
• Particle Swarm Optimization
• Tribes
• Clustering
• Outliers