Reinforcement Learning Based Peer-to-Peer Energy Trade Management Using Community Energy Storage in Local Energy Market

Abstract

Many studies have proposed a peer-to-peer energy market where the prosumers’ actions, including energy consumption, charge and discharge schedule of energy storage systems, and transactions in local energy markets, are controlled by a central operator. In this paper, prosumers’ actions are not controlled by an operator, and the prosumers freely participate in the local energy market to trade energy with other prosumers. We designed and modeled a local energy market with a management algorithm that uses community energy storage for prosumers who competitively participate in trade in the real-time energy market. We propose an energy-trade management algorithm that manages the trades of prosumers in two phases based on bids and offers submitted by prosumers. The first phase is to manage the trade of prosumers who have submitted fair prices to trade with other prosumers in the real-time energy market. The second phase is managing the trade of prosumers that could not trade in the first phase. Community energy storage is employed in the second phase and controlled by a reinforcement learning-based trading algorithm to decide whether to buy, sell, or do nothing with the prosumers. The action of buying and selling means charging and discharging the community energy storage, respectively. Numerical results show that the proposed trading algorithm gains a near-maximum profit. Besides, we verified that community energy storage yields more profit than the battery wear-out cost.

Keywords

• smart grid
• local energy market
• peer-to-peer energy trade
• community energy storage
• reinforcement learning