RL-AKF: An Adaptive Kalman Filter Navigation Algorithm Based on Reinforcement Learning for Ground Vehicles

Abstract

Kalman filter is a commonly used method in the Global Navigation Satellite System (GNSS)/Inertial Navigation System (INS) integrated navigation system, in which the process noise covariance matrix has a significant influence on the positioning accuracy and sometimes even causes the filter to diverge when using the process noise covariance matrix with large errors. Though many studies have been done on process noise covariance estimation, the ability of the existing methods to adapt to dynamic and complex environments is still weak. To obtain accurate and robust localization results under various complex and dynamic environments, we propose an adaptive Kalman filter navigation algorithm (which is simply called RL-AKF), which can adaptively estimate the process noise covariance matrix using a reinforcement learning approach. By taking the integrated navigation system as the environment, and the opposite of the current positioning error as the reward, the adaptive Kalman filter navigation algorithm uses the deep deterministic policy gradient to obtain the most optimal process noise covariance matrix estimation from the continuous action space. Extensive experimental results show that our proposed algorithm can accurately estimate the process noise covariance matrix, which is robust under different data collection times, different GNSS outage time periods, and using different integration navigation fusion schemes. The RL-AKF achieves an average positioning error of 0.6517 m within 10 s GNSS outage for GNSS/INS integrated navigation system and 14.9426 m and 15.3380 m within 300 s GNSS outage for the GNSS/INS/Odometer (ODO) and the GNSS/INS/Non-Holonomic Constraint (NHC) integrated navigation systems, respectively.

Keywords

• integrated navigation
• Kalman filter
• process noise covariance estimation
• reinforcement learning
• deep deterministic policy gradient