Bayesian neural network with efficient priors for online quality prediction

Abstract

In process industries, neural networks are popular models for quality prediction due to their superiorities on dealing with nonlinearity and high dimensionality of process data. However, small sample size of online data will result in epistemic uncertainty of prediction model, so neural networks with fixed parameters are prone to overfitting and provide over-confident predictions with poor robustness. To this end, we propose a Pior-Assistant Random Bayesian Neural Network (PA-RBNN). First, the Random Vector Functional Link Neural Network (RVFLNN) with Bayes backpropagation (RBNN) is introduced to the last layer of the off-lined deep neural network model. The prior mean values and variances of parameters are determined by the closed-form solution from RVFLNN. Then, the semi-supervised form of PA-RBNN is further developed by introducing manifold regularization to RVFLNN to obtain the semi-supervised prior means and variances of parameters. In this way, the proposed prior provides a reasonable range before learning the parameter posterior, and both the labeled and unlabeled data are utilized to obtain the prior. The proposed method is verified by the benchmarked industrial process data. The experimental results show that the assistant prior has an optimistic effect on guiding the training process towards the basins of attraction of minima with less computation cost, and improves the generalization of prediction model.

Keywords

• Online quality prediction
• Epistemic uncertainty
• Bayesian neural network
• Prior-assistant
• Semi-supervised learning