Towards explicit representation of an artificial neural network model: Comparison of two artificial neural network rule extraction approaches

Abstract

In the quest for interpretable models, two versions of a neural network rule extraction algorithm were proposed and compared. The two algorithms are called the Piece-Wise Linear Artificial Neural Network (PWL-ANN) and enhanced Piece-Wise Linear Artificial Neural Network (enhanced PWL-ANN) algorithms. The PWL-ANN algorithm is a decomposition artificial neural network (ANN) rule extraction algorithm, and the enhanced PWL-ANN algorithm improves upon the PWL-ANN algorithm and extracts multiple linear regression equations from a trained ANN model by approximating the hidden sigmoid activation functions using N-piece linear equations. In doing so, the algorithm provides interpretable models from the originally trained opaque ANN models. A detailed application case study illustrates how the generated enhanced-PWL-ANN models can provide understandable IF-THEN rules about a problem domain. Comparison of the results generated by the two versions of the PWL-ANN algorithm showed that in comparison to the PWL-ANN models, the enhanced-PWL-ANN models support improved fidelities to the originally trained ANN models. The results also showed that more concise rule sets could be generated using the enhanced-PWL-ANN algorithm. If a more simplified set of rules is desired, the enhanced-PWL-ANN algorithm can be combined with the decision tree approach. Potential application of the algorithms to domains related to petroleum engineering can help enhance understanding of the problems.

Keywords

• Artificial neural networks
• Rule extraction
• Regression problem
• Algorithm design