A New Deep Q-Network Design for QoS Multicast Routing in Cognitive Radio MANETs

Abstract

In this paper, we propose a new deep Q-network (DQN) design for quality-of-service (QoS) multicast routing (DQMR) protocol to establish efficient QoS multicast (EQM) trees in cognitive radio mobile ad hoc networks (CR-MANETs). An EQM tree is a shortest-path multicast tree with minimum end-to-end (E2E) cost (a combination of queuing size ratio and link stability) subject to QoS constraints such as queuing size ratio, link stability, number of hops, number of time slots and avoiding the licensed channel of primary users. Particularly, we propose an NP-complete optimization problem such that its feasible solution is an EQM tree. To address this problem, we design a new DQN model and a new game-based model to form EQM trees in real-time by offline training instead of online training as done in previous papers. Moreover, the DQMR protocol is also guaranteed to have high stability, low routing delay, low control overhead, and high packet delivery ratio (PDR). Furthermore, one more new contribution of the paper is that exact closed-form expressions for the E2E queuing delay of a multicast routing tree are also derived assuming random waypoint mobility and the reference point group mobility models to compare with simulation results of routing delay. Simulation results show that the DQMR protocol outperforms multicast ad hoc on-demand distance vector routing protocol in terms of routing delay, control overhead, and PDR.

Keywords

• Cognitive mobile ad hoc networks
• cross-layer
• deep Q-network
• game theory
• QoS multicast routing