A Q-learning-based smart clustering routing method in flying Ad Hoc networks

Abstract

Flying ad hoc networks (FANETs) have particular importance in various military and civilian applications due to their specific features, including frequent topological changes, the movement of drones in a three-dimensional space, and their restricted energy. These features have created challenges for designing cluster-based routing protocols. In this paper, a Q-learning-based smart clustering routing method (QSCR) is suggested in FANETs. In QSCR, each node discovers its neighbors through the periodic exchange of hello messages. The hello time interval is different in each cluster, and cluster leaders determine this interval based on the average speed similarity. Next, an adaptive clustering process is presented for categorizing drones in the clusters. In this step, the cluster leader is selected based on a new parameter called merit value, which includes residual energy, centrality, neighbor degree, speed similarity, and link validity time. Then, a centralized Q-learning model is presented to tune weight coefficients related to merit parameters dynamically. In the last step, the routing process is done using a greedy forwarding technique. Finally, QSCR is run on NS2, and the simulation results of QSCR are compared with those of ICRA, WCA, and DCA. These results show that QSCR carries out the clustering process rapidly but has less cluster stability than ICRA. QSCR gets energy efficiency and improves network lifetime. In the routing process, QSCR has a high packet delivery rate compared to other schemes. Also, the number of isolated clusters created in QSCR is less than other clustering methods. However, the proposed scheme has a higher end-to-end delay than ICRA. Also, this scheme experiences more communication overhead than ICRA slightly.

Keywords

• Flying ad hoc networks (FANETs)
• Clustering
• Unmanned aerial vehicles (UAVs)
• Reinforcement learning (RL)
• Machine learning (ML)