Adaptive Modular Reinforcement Learning for Robot Controlled in Multiple Environments

Abstract

This paper proposes an adaptive modular reinforcement learning architecture and an algorithm for robot control operating in multiple environments. Reinforcement learning autonomously acquires control rules by interacting between the agent and the controlled system. Consequently, reinforcement learning is expected to be applied to robot control where model identification is difficult. These robots are often expected to operate in multiple environments. However, existing reinforcement learning algorithms require prior knowledge of changes in the environment. In this paper, we proposed an architecture and algorithm that does not require prior knowledge of the environment. In this architecture, the policy can be acquired by increasing the number of modules based on the interaction with the controlled system. Therefore, the proposed method can be applied to robots whose dynamics change without losing the feature that the reinforcement learning algorithm does not require prior knowledge of the controlled system. Two numerical experiments were conducted to evaluate the proposed method, which improved the performance by approximately 25 % when compared to the conventional methods.

Keywords

• Multiple environments
• nonstationary environment
• reinforcement learning