Integral reinforcement learning based dynamic stackelberg pursuit-evasion game for unmanned surface vehicles

Abstract

The dynamic stackelberg pursuit-evasion (PE) game of unmanned surface vehicles (USVs) is discussed in this paper. The optimal solution method of the USVs’ PE game is proposed. The USVs’ PE game is firstly described by the pursuit motion on two-dimensional bounded surface, and considering the optimal decision order of evader and pursuer, the pursuit-evasion game is modeled by a dynamic stackelberg game. Then the optimal game solution problem is transformed into the solution of Hamilton–Jacobi–Isaacs equations, and on-policy iteration of integral reinforcement learning algorithm is utilized. Neural network is also utilized for the value iteration solution of the dynamic stackelberg pursuit-evasion game. The existence of the stackelberg equilibrium and the global asymptotic stability of the system are all discussed related to optimal control and differential game. Finally, the presented method is tested by pursuit-evasion game between USVs.

Keywords

• Dynamic stackelberg game
• Integral reinforcement learning
• Pursuit-evasion game
• Policy iteration