Research on recurrent neural network model based on weight activity evaluation

Abstract

Given the complex structure and parameter redundancy of recurrent neural networks such as LSTM, related research and analysis on the structure of recurrent neural networks have been done. To improve the structural rationality of the recurrent neural network and reduce the amount of calculation of network parameters, a weight activity evaluation algorithm is proposed that evaluates the activity of the basic unit of the network. Through experiments and tests on arrhythmia data, the differences in the weight activity of the LSTM network and the change characteristics of weights and gradients are analyzed. The experimental results show that this algorithm can better optimize the recurrent neural network structure and reduce the redundancy of network parameters.

Keywords

• weight activity evaluation
• parameter redundancy
• recurrent neural network