Cascaded Deep Reinforcement Learning-Based Multi-Revolution Low-Thrust Spacecraft Orbit-Transfer

Abstract

Transferring an all-electric spacecraft from a launch injection orbit to the geosynchronous equatorial orbit (GEO) using a low thrust propulsion system presents a significant challenge due to the long transfer time typically spanning several months. To address the challenge of determining such long time-scale orbit-raising maneuvers to GEO, this paper presents a novel technique to compute transfers starting from geostationary transfer orbit (GTO) and super-GTO. The transfer is complex, involving multiple eclipses and revolutions. To tackle this challenge, we introduce a cascaded deep reinforcement learning (DRL) model to guide a low-thrust spacecraft towards the desired orbit by determining an appropriate thrust direction at each state. To ensure mission requirements, a gradient-aided reward function incorporating the orbital elements, guides the DRL agent to obtain the optimal flight time. The obtained results demonstrate that our proposed approach yields optimal or near-optimal time-efficient spacecraft orbit-raising. DRL implementation is important for spacecraft autonomy; in this context, we demonstrate that our DRL-based trajectory planning provides significantly better transfer time as compared to state-of-the-art approaches that allow for automated trajectory computation.

Keywords

• Deep reinforcement learning
• cascaded reinforcement learning
• soft actor-critic algorithm
• spacecraft orbit-transfer
• solar-electric propulsion
• optimization