Applying Deep Reinforcement Learning to Algorithmic Trading

Abstract

At the moment, there is a large volume of literature on exchange trading. Obviously, every year the mathematical base of work is becoming more complicated along with an increase in computing power, machines can process more metrics from year to year and produce more accurate solutions per unit of time. The use of deep learning has already proven itself well, as the application of this approach has given a quantum leap in algorithmic trading. The article presents an algorithm for trading long contracts with one asset in the financial market in the Python programming language using the LSTM neural network using the Keras library, which is used as a demo example in the Reinforcement Learning discipline. The formalized LSTM model solves the vanishing gradient problem, which can hold the gradient of the objective function relative to the state signal. As applied to our problem, such an improvement in the model allows us to collect data on certain patterns of price changes, that is, when predicting the price of the next step, we rely not only on the data of the previous step, but also on earlier data, when there was a similar state of the environment. Sharpe Ratio is used to determine the optimal strategy and make decisions at each time of application. The optimal minimum time period for the model operation has been determined; the signal transmission delay from the moment the market situation changes until the signal is received by the model, which will be infinitely small, and the computing power will be considered infinitely large. These assumptions give the right to say: when the market situation changes, the model is instantly ready to react and make a decision to sell, buy or hold an asset.

Keywords

• algorithmic trading
• deep learning
• reinforcement learning
• recurrent neural networks
• lstm model
• demo example