Fuzzy Reinforcement Learning for Robust Spectrum Access in Dynamic Shared Networks

Abstract

The persistent increases of wireless terminals have brought about diverse shared networks, where robust and efficient spectrum reuse among heterogeneous users is of critical importance while still remains as a challenging task for practical application. In this paper, we study the problem of robust spectrum access (RSA) in a canonical wireless shared network (WSN) with fully considering the inherent dynamics of the wireless environment. The non-static features of WSNs result in uncertain channel state information (CSI) and complicated coupling interference, which can't be directly formulated as the well-accepted crisp game model, rendering most existing perfect CSI relied approaches inefficient or even unfeasible. To address this, by interpreting the estimated CSI with uncertainty as fuzzy number, a novel framework referred to as a non-cooperative fuzzy game (NC-FG) is adopted, whereby the user utility is mapped as a fuzzy value via the user-defined fuzzy utility function. Based on the derived property of the NC-FG that fuzzy Nash equilibrium (FNE) exists, a fuzzy-logic inspired reinforcement learning (FLRL) algorithm is proposed to achieve the FNE solutions of the constructed NC-FG to obtain the RSA in dynamic WSN, with which both the iterative learning and decision making procedures are implemented in a fuzzy-space, thus the sensitiveness of our scheme to environmental variations is alleviated. Finally, numerical simulations are provided to demonstrate the convergence, effectiveness, and superiority of our proposed FLRL algorithm in dynamic WSNs.

Keywords

• Dynamic wireless shared network
• robust spectrum access
• fuzzy space
• non-cooperative fuzzy game
• fuzzy-logic based reinforcement learning