Intelligent Attitude Control of Hypersonic Vehicle Based on DDQN and Deep Q-Learning from Demonstrations

Abstract

In order to improve the speed and accuracy of solving the attitude control problem of hypersonic vehicle, an intelligent attitude control method of hypersonic vehicle based on demonstration learning is proposed. Firstly, the control model of hypersonic vehicle is established, and the appropriate action is selected as the attitude control output. Secondly, an algorithm based on DDQN (Double Deep Q-Network) and DQfD (Deep Q-learning from Demonstrations) is designed, which divides the training of agents into two stages: pre-training and formal training. In the pre-training stage, the agent extracts small batch data from the demonstration data, and applies four loss functions to update the neural network. In the formal training phase, samples are taken from the data generated by its own training and demonstration data, and the proportion of two types of data in each small batch is controlled through priority experience replay buffer. Learning through interaction with the environment, so that the hypersonic vehicle can adaptively adjust its attitude according to changes in the flight environment. The simulation results show that the reinforcement learning method based on demonstration data can track control command, realize the attitude control of hypersonic vehicle, and improve the performance of neural network in the early stage of training, with a higher average reward.

Keywords

• |hypersonic vehicle|attitude control|reinforcement learning|learning from demonstrations|ddqn