Does Adding of Neurons to the Network Layer Lead to Increased Transmission Efficiency?

Abstract

The aim of this study is to contribute to the important question in Neuroscience of whether the number of neurons in a given layer of a network affects transmission efficiency. Mutual Information, as defined by Shannon, between the input and output signals for certain classes of networks is analyzed theoretically and numerically. A Levy-Baxter probabilistic neural model is applied. This model includes all important qualitative mechanisms involved in the transmission process in the brain. We derived analytical formulas for the Mutual Information of input signals coming from Information Sources as Bernoulli processes. These formulas depend on the parameters of the Information Source, neurons and network. Numerical simulations were performed using these equations. It turned out, that the Mutual Information starting from a certain value increased very slowly with the number of neurons being added. The increase is of the rate $m^{-c}$ where $m$ is the number of neurons in the transmission layer, and $c$ is very small. The calculations also show that for a practical number (up to 15000) of neurons, the Mutual Information reaches only approximately half of the information that is carried out by the input signal. The influence of noise on the transmission efficiency depending on the number of neurons was also analyzed. It turned out that the noise level at which transmission is optimal increases significantly with this number. Our results indicate that a large number of neurons in the network does not mean an essential improvement in transmission efficiency, but can contribute to reliability.

Keywords

• Shannon communication theory
• neural network
• network layer
• transmission efficiency
• mutual information
• model of neuron