Hidden-layer configurations in reinforcement learning models for stock portfolio optimization

Abstract

In the rapidly evolving field of artificial intelligence and financial markets, efficient and adaptive portfolio management strategies are becoming increasingly critical. This study explores the impact of hidden-layer configurations in reinforcement learning models for stock portfolio optimization. Using a portfolio of 45 actively traded stocks in the Indonesian stock market, the performance of four reinforcement learning algorithms—Advantage Actor-Critic (A2C), Deep Deterministic Policy Gradient (DDPG), Proximal Policy Optimization (PPO), and Twin Delayed Deep Deterministic Policy Gradient (TD3)—is evaluated with zero, one, and two hidden layers. The results show that A2C and DDPG models perform effectively without hidden layers, with A2C achieving the highest Cumulative Return (CuR) and Annualized Return Rate (ARR) among all configurations. Adding hidden layers to A2C improved risk management, resulting in a lower Maximum Drawdown (MDD) and a higher Annualized Sharpe Ratio (ASR). DDPG exhibited consistently strong performance, with its zero hidden-layer model showing the highest ASR. Conversely, PPO underperformed across all configurations, with negative returns in the zero-layer setup and marginal improvements with added complexity. Introducing additional hidden layers improved TD3′s performance, enhancing risk-adjusted returns. These findings suggest that the effectiveness of hidden-layer configurations depends on the specific algorithm used. While A2C and DDPG benefit from increased complexity, simpler architectures may be more suitable for PPO and TD3. This study offers new insights into optimizing reinforcement learning models for stock portfolio management by adjusting hidden-layer structures to balance returns and risk.

Keywords

• Reinforcement learning
• Stock portfolio
• Algorithmic trading
• Financial market
• Stock return
• Neural network