Sim-to-Real Transfer Reinforcement Learning for Position Control of Pneumatic Continuum Manipulator

Abstract

Reinforcement learning (RL) is attempted to be applied to the control of continuum robots. Because of the inefficiency and high cost of collecting samples in the real world, the control strategy is usually learned in simulation. However, due to the gap between the simulation and the real world, the performance of the strategy learned in the simulation will be reduced when it is transferred to the real world. This paper proposes a strategy learning and Simulation-to-Real (Sim-to-Real) transfer framework for the position control of pneumatic continuum manipulator (PCM). The dynamics model of the PCM, which is used as the simulation environment of RL, is represented by long short-term memory (LSTM). The probabilistic inference and learning for control (PILCO) is used to train the control strategy. In order to utilize the information of the strategy learned in simulation, the Sim-to-Real transfer method based on strategy fine-tuning is proposed. By fine-tuning the strategy, the strategy learned in simulation can be applied to the real world. Finally, an experiment is carried out on a PCM to verify the effectiveness of the proposed framework.

Keywords

• Pneumatic continuum manipulator
• position control
• PILCO reinforcement learning
• simulation-to-real