Imitation Reinforcement Learning-Based Remote Rotary Inverted Pendulum Control in OpenFlow Network

Abstract

Rotary inverted pendulum is an unstable and highly nonlinear device and has been used as a common application model in nonlinear control engineering field. In this paper, we use a rotary inverted pendulum as a deep reinforcement learning environment. The real device is composed of a cyber environment and physical environment based on the OpenFlow network, and the MQTT protocol is used on the Ethernet connection to connect the cyber environment and the physical environment. The reinforcement learning agent is learned to control the real device located remotely from the controller, and the classical PID controller is also utilized to implement the imitation reinforcement learning and facilitate the learning process. From our CPS-based experimental system, we verify that a deep reinforcement learning agent can successfully control the real device located remotely from the agent, and our imitation learning strategy can make the learning time reduced effectively.

Keywords

• Reinforcement learning
• remote control
• control engineering
• OpenFlow
• CPS