Q-Learning Based Autonomous Control of the Auxiliary Power Network of a Ship

Abstract

We present a reinforcement learning (RL) model that is based on Q-learning for the autonomous control of ship auxiliary power networks. The development and application of the proposed model is demonstrated using a case-study ship as the platform. The auxiliary power network of the ship is represented as a Markov Decision Process (MDP). Q-learning is then used to teach an agent to operate in this MDP by choosing actions in each operating state which would minimize fuel consumption while also respecting the boundary conditions of the network. The presented work is based on an extensive data set received from one of the cruise-line operators on the Baltic Sea. This data set was preprocessed to extract information for the state representation of the auxiliary network, which was used for training and validating the model. As a result, it is shown that the developed method produces an autonomous control policy for the auxiliary power network that outperforms the current human operated manual control of the case-study ship. An average of 0.9 % fuel oil savings are attained over the analyzed round-trips with control that displayed similar robustness against blackouts as the current operation of the ship. This amounts to 32 tons of fuel oil saved annually. In addition, it is shown that the developed model can be reconfigured for different levels of robustness, depending on the preferred trade-off between maintained reserve power and fuel savings.

Keywords

• Autonomous shipping
• energy consumption reduction
• Ferry
• machinery
• Q-learning
• reinforcement learning