Deep Reinforcement Learning Heterogeneous Channels for Poisson Multiple Access

Abstract

This paper proposes a medium access control (MAC) protocol based on deep reinforcement learning (DRL), i.e., multi-channel transmit deep-reinforcement learning multi-channel access (MCT-DLMA) in heterogeneous wireless networks (HetNets). The work concerns practical unsaturated channel traffic that arrives following a Poisson distribution instead of saturated traffic that arrives before.By learning the access mode from historical information, MCT-DLMA can well fill the spectrum holes in the communication of existing users. In particular, it enables the cognitive user to multi-channel transmit at a time, e.g., via the multi-carrier technology. Thus, the spectrum resource can be fully utilized, and the sum throughput of the HetNet is maximized. Simulation results show that the proposed algorithm provides a much higher throughput than the conventional schemes, i.e., the whittle index policy and the DLMA algorithms for both the saturated and unsaturated traffic, respectively. In addition, it also achieves a near-optimal result in dynamic environments with changing primary users, which proves the enhanced robustness to time-varying communications.

Keywords

• deep reinforcement learning
• MAC protocol
• heterogeneous network
• spectrum hole