Reinforcement learning for battery energy management: A new balancing approach for Li-ion battery packs

Abstract

This study investigates the challenge of cell balancing in battery management systems (BMS) for lithium-ion batteries. Effective cell balancing is crucial for maximizing the usable capacity and lifespan of battery packs, which is essential for the widespread adoption of electric vehicles and the reduction of greenhouse gas emissions. A novel deep reinforcement learning (deep RL) approach is proposed for passive balancing with switched shunt resistors. Notable deep RL algorithms capable of handling discrete actions, such as Trust Region Policy Optimization (TRPO), Proximal Policy Optimization (PPO), Deep Q-Network (DQN), Augmented Random Search (ARS), and Asynchronous Advantage Actor Critic (A3C), are investigated. TRPO demonstrates superior performance compared to other deep RL algorithms and rule-based methods in both charging and discharging scenarios without requiring fine-tuning, optimizing the balance between cell balancing and switch changes. It achieves up to 16.8% improvement in battery pack capacity, 69.4% reduction in state-of-charge variance among cells, and 40.4% decrease in the number of switching operations in simulation results for five li-ion cells connected in series.The study introduces an innovative application of deep RL for passive balancing, a comprehensive battery cell modeling technique, and a tailored multi-objective reward function that balances cell balancing and switching costs. This work represents a significant advancement in applying deep RL to battery management systems, providing a framework for further research and practical implementation in energy storage systems.

Keywords

• Li-ion battery
• Passive balancing
• Deep reinforcement learning
• Battery management system