Reinforcement Learning Based MAC Protocol (UW-ALOHA-QM) for Mobile Underwater Acoustic Sensor Networks

Abstract

The demand for ocean exploration and exploitation is rapidly increasing and this has led to rapid growth in the market of mobile vehicles. Given the mobility, the key challenge is to design a highly adaptive solution with minimal signalling (and the associated delays) which current techniques have not fully addressed. Therefore, the mobility and associated challenges in the underwater channel necessitates the design of a new approach to Medium Access Control (MAC) which provides the capability to adapt to rapidly changing conditions with no reliance on signalling which causes delays. This paper proposes the UW-ALOHA-QM protocol, which uses reinforcement learning to allow nodes to adapt to the time varying environment through trial-and-error interaction and thereby improve network resilience and adaptability. Simulations are carried out in four distinct scenarios in which node mobility patterns are significantly different. Simulation results demonstrate that UW-ALOHA-QM provides up to 300% improvement in channel utilisation with respect to existing protocols designed for mobile networks.

Keywords

• Medium access control
• mobile sensor networks
• reinforcement learning
• Q-learning
• underwater acoustic networks