Stepwise Regression for Increasing the Predictive Accuracy of Artificial Neural Networks: Applications in Benchmark and Advanced Problems

Abstract

A new technique is proposed to increase the prediction accuracy of artificial neural networks (ANNs). This technique applies a stepwise regression (SR) procedure to the input data variables, which adds nonlinear terms into the input data in a way that maximizes the regression between the output and the input data. In this study, the SR procedure adds quadratic terms and products of the input variables on pairs. Afterwards, the ANN is trained based on the enhanced input data obtained by SR. After testing the proposed SR-ANN algorithm in four benchmark function approximation problems found in the literature, six examples of multivariate training data are considered, of two different sizes (big and small) often encountered in engineering applications and of three different distributions in which the diversity and correlation of the data are varied, and the testing performance of the ANN for varying sizes of its hidden layer is investigated. It is shown that the proposed SR-ANN algorithm can reduce the prediction error by a factor of up to 26 and increase the regression coefficient between predicted and actual data in all cases compared to ANNs trained with ordinary algorithms.

Keywords

• artificial neural networks
• universal approximation theorem
• stepwise regression