Graph Pointer Network Based Hierarchical Curriculum Reinforcement Learning Method Solving Shuttle Tankers Scheduling Problem

Abstract

Shuttle tankers scheduling is an important task in offshore oil and gas transportation process, which involves operating time window fulfillment, optimal transportation planning, and proper inventory management. However, conventional approaches like Mixed Integer Linear Programming (MILP) or meta heuristic algorithms often fail in long running time. In this paper, a Graph Pointer Network (GPN) based Hierarchical Curriculum Reinforcement Learning (HCRL) method is proposed to solve Shuttle Tankers Scheduling Problem (STSP). The model is trained to divide STSP into voyage and operation stages and generate routing and inventory management decisions sequentially. An asynchronous training strategy is developed to address the coupling between stages. Comparison experiments demonstrate that the proposed HCRL method achieves 12% shorter tour lengths on average compared to heuristic algorithms. Additional experiments validate its generalizability to unseen instances and scalability to larger instances.

Keywords

• graph pointer network
• hierarchical reinforcement learning
• curriculum learning
• shuttle tanker scheduling problem