Efficient Implementation of Multilayer Perceptrons: Reducing Execution Time and Memory Consumption

Abstract

A technique is presented that reduces the required memory of neural networks through improving weight storage. In contrast to traditional methods, which have an exponential memory overhead with the increase in network size, the proposed method stores only the number of connections between neurons. The proposed method is evaluated on feedforward networks and demonstrates memory saving capabilities of up to almost 80% while also being more efficient, especially with larger architectures.

Keywords

• neural networks
• multilayer perceptron
• compressed weight matrix
• weight density
• sparsity