Key Laboratory of Remote Measurement and Control of Jiangsu Province, School of Instrument Science and Engineering, Southeast University, Nanjing, China
Yuan Yang
Key Laboratory of Remote Measurement and Control of Jiangsu Province, School of Instrument Science and Engineering, Southeast University, Nanjing, China
Key Laboratory of Remote Measurement and Control of Jiangsu Province, School of Instrument Science and Engineering, Southeast University, Nanjing, China
Guochen Pang
School of Automation and Electrical Engineering, Linyi University, Linyi, China
The existing overlap singularities in the parameter space significantly affect the learning dynamics of the multilayer perceptrons. From the obtained theoretical learning trajectories near overlap singularity, when the learning process has been affected by the overlap singularity, the influence area of the overlap singularity is just the line space where the two hidden units equal to each other. However, in the practical applications, different case has been observed and the influence area of such singularity may be larger. By analyzing the generalization error of multilayer perceptrons, we find that the error surface is much flatter near overlap singularity and the singularity would have much larger influence area. Finally, the validity of the obtained results are verified by taking an artificial experiment and two real-data experiments.
