MARL-TS Method for Underwater Acoustic Networks in Time-Varying Channels

Abstract

Underwater acoustic communication faces numerous challenges in transmission scheduling and decision-making due to its high propagation delay, time-varying channel characteristics, and limited bandwidth. To enhance communication efficiency in complex underwater acoustic environments, this paper proposed a multi-agent reinforcement learning(MARL)-based cross-layer transmission scheduling(TS) method for underwater acoustic networks, termed MARL-TS. This method addressed the high propagation delay and dynamic channel environments by leveraging transmission node buffer states and channel conditions as the foundation while optimizing transmission efficiency and transmission delay of the communication network. It adaptively performs cross-layer optimization to jointly optimize power allocation and timeslot resource scheduling. To learn the optimal transmission strategy, this paper constructed a learnable policy network and a value network, integrating multi-agent cooperative learning to improve strategy optimization efficiency and adaptive decision-making capabilities. Simulation results demonstrate that compared with existing reinforcement learning-based multiple access control(MAC) protocols, MARL-TS significantly enhances transmission efficiency and reduces transmission delay. Notably, it exhibits superior adaptability and stability in multi-node and high-load scenarios, offering a novel approach for optimizing complex underwater communication systems.

Keywords

• underwater acoustic network
• time-varying channel
• multi-agent reinforcement learning
• cross-layer transmission scheduling