A convergent Deep Learning algorithm for approximation of polynomials

Abstract

We start from the contractive functional equation proposed in [4], where it was shown that the polynomial solution of functional equation can be used to initialize a Neural Network structure, with a controlled accuracy. We propose a novel algorithm, where the functional equation is solved with a converging iterative algorithm which can be realized as a Machine Learning training method iteratively with respect to the number of layers. The proof of convergence is performed with respect to the $L^\infty $ norm. Numerical tests illustrate the theory and show that stochastic gradient descent methods can be used with good accuracy for this problem.