Reinforcement learning-based automated modulation switching algorithm for an enhanced underwater acoustic communication

Abstract

Acoustic communication is the preferred method for underwater communication due to its superior propagation characteristics, including longer range, lower attenuation, better penetration, and adaptability to water density. However, challenges like varying water depths, temperature gradients, noise, and multipath propagation require innovative solutions beyond fixed-data rate systems and single modulation schemes. To address these challenges, we propose a reinforcement learning (RL)-based automated modulation switching algorithm designed to enhance data transmission efficiency. This approach leverages the UNETStack software and the Frequency Hopping-Binary Frequency Shift Key (FH-BFSK) modulation, extending its capabilities by incorporating Amplitude Shift Keying (ASK), Phase Shift Keying (PSK), and Orthogonal Frequency Division Multiplexing (OFDM) modulation techniques. We designed a cost-effective, software-controlled acoustic modem using commercial off-the-shelf (COTS) components, enabling full-duplex underwater communication. The RL algorithm utilizes a Q-matrix guided by a greedy policy to assess network conditions, such as data volume and data rates. It continuously monitors underwater environments to select the most suitable modulation scheme dynamically. Experimental validation demonstrates a 3.648 % improvement in Received Signal Strength Indicator (RSSI), a 32 % reduction in bit error rate at a constant 7 dB Signal-to-Noise Ratio (SNR), and a 5 % increase in utility at 10 dB SNR when using OFDM selected by the reinforcement learning algorithm, compared to FH-BFSK.

Keywords

• Reinforcement learning
• OFDM
• FSK
• UNET
• Acoustic modem
• Underwater