Configurable Model for Sigmoid and Hyperbolic Tangent Functions

Abstract

Recurrent neural networks (RNNs) are considered to be among the most important types of neural networks especially for the applications where processing of a sequence of data comes to place. RNNs are in general computationally expensive and need a lot of processing time and power. Therefore, there is a strong need to reduce the processing time to be able to use them in an embedded environment with limited resources. In this work, we present an accelerated field programmable gate array (FPGA) model for RNNs with an emphasis on long short-term memory neural networks (LSTMs). A new configurable block capable of calculating Tanh and Sigmoid activation functions is proposed and analyzed. The solution is based on a look-up table and additional simple math operations, which leads to a speedup of the proposed model of the neural network. Results are obtained and compared with other work by the simulation tool ISE Xillinx.

Keywords

• activation functions
• deep learning
• fpga
• lstm
• recurrent neural network