High-Reliability Multi-Agent Q-Learning-Based Scheduling for D2D Microgrid Communications

Abstract

This paper proposes a multi-agent Q-learning-based resource allocation algorithm that allows long-term evolution (LTE)-enabled device-to-device (D2D) communication agents to generate the orthogonal transmission schedules outside the network coverage. This algorithm reduces packet drop rates (PDR) in distributed D2D communication networks to meet the quality-of-service requirements of the microgrid communications. The data traffic characteristics of three archetypal smart grid applications, namely demand response, solar, and generation forecasting, and synchrophasor communications, were simulated under seven different traffic congestion scenarios, where the total aggregate throughput of users ranged from 50% to 140% channel utilization. The PDR and latency performance of the proposed algorithm were compared with the existing random self-allocation mechanism introduced under the Third-Generation Partnership Project's LTE Release 12 standard for such scenarios. Our algorithm outperformed the LTE algorithm for all tested scenarios, demonstrating 20%-40% absolute reductions in PDR and 10-20-ms reductions in latency for all microgrid applications. The use of our algorithm in a simulated D2D-enabled demand response application resulted in a hundredfold reduction in power oscillations about the desired power flows.

Keywords

• Device-to-device communications
• Q-learning
• reliability
• resource allocation
• smart grid