Extending the BESS Lifetime: A Cooperative Multi-Agent Deep Q Network Framework for a Parallel-Series Connected Battery Pack

Abstract

In this paper, we propose a battery management algorithm to maximize the lifetime of a parallel-series connected battery pack with heterogeneous states of health in a battery energy storage system. The growth of retired lithium-ion batteries from electric vehicles increases the applications for battery energy storage systems, which typically group multiple individual batteries with heterogeneous states of health in parallel and series to achieve the required voltage and capacity. However, previous work has primarily focused on either parallel or series connections of batteries due to the complexity of managing diverse battery states, such as state of charge and state of health. To address the scheduling in parallel-series connections, we propose a cooperative multi-agent deep Q network framework that leverages multi-agent deep reinforcement learning to observe multiple states within the battery energy storage system and optimize the scheduling of cells and modules in a parallel-series connected battery pack. Our approach not only balances the states of health across the cells and modules but also enhances the overall lifetime of the battery pack. Through simulation, we demonstrate that our algorithm extends the battery pack’s lifetime by up to 16.27% compared to previous work and exhibits robustness in adapting to various power demand conditions.

Keywords

• battery energy storage system
• retired lithium-ion batteries
• battery management system
• deep reinforcement learning
• multi-agent
• cooperative learning