Alexandria Engineering Journal (Mar 2025)
Multiagent deep reinforcement learning based Energy efficient resource management scheme for RIS assisted D2D users in 6G-aided smart cities environment
Abstract
Device-to-device communication (D2D-C) is one of the promising technologies for the sixth-generation (6G) environment. This is because it enhances end-user throughput, energy efficiency (EE), and the network’s quality of service (QoS) even when the users are in complex networks or high-traffic zones of the smart cities. However, in D2D-C, different channels share the same subchannels (SCs), which causes considerable interference to cellular links. Moreover, ultra-massive connectivity (UMC) is a significant challenge in this environment. To overcome these obstacles, we paired the unmanned aerial vehicle (UAV) with the power domain non-orthogonal multiple access (PD-NOMA) technology to improve coverage and connection while reducing interference. Also, we used the reconfigurable intelligent surfaces (RISs) for propagation between UAV and D2D pairs (D2DPs) as they do not require much energy resources, due to which EE increases. Then, we propose a methodology for energy-efficient allocation of resources in RIS-assisted NOMA-enabled underlaying UAVs to D2D users. To achieve this goal, we first use the Markov decision process (MDP) to transform the formulated problem into a machine-learning form using the reinforcement technique. A multi-agent, priority sampling-based, decentralized and coordinated, dueling deep Q-network (PS-DC-DDQN) technique is proposed since the network is complicated with large state and action spaces. To reduce the complexity, the data for resource allocation and power is distributed among neighboring agents only in a decentralized and coordinated manner. Moreover, to optimize the RIS phase shift, the centralized-DDQN (C-DDQN) algorithm is recommended to reduce the power consumption. Simulation results demonstrated that the suggested PS-DC-DDQN algorithm has 7.3%, 17.07%, and 29.26% higher EE in comparison to the state-of-the-art FA-DDQN, DDQN, and DQN techniques, respectively.
