Skill acquisition and transfer through common subgoals

Abstract

The main challenge of using robots in social environments such as houses is coping with the frequent changes in tasks. Since it is infeasible to come up with an implementation for all possible cases of all tasks, robots should find solutions for new problems by themselves. So, learning is one of the major abilities for a robot to deal with changing tasks. However, it is generally time-consuming to find even a near-optimal solution for complex tasks through learning. On the other hand, learning in humans is a never-ending process and much faster, thanks to transferring prior knowledge. In this work, we build a knowledge base (called as skill library) from the subsets of the tasks discovered during the learning process. Since most of the tasks encountered have common subsets, the skill library enables us to transfer previous experiences while learning the strategy of a new task. The robot progressively accumulates skills to reduce the difficulty of learning the forthcoming tasks. We choose navigation in different unknown environments as the test bed. The results show a significant improvement especially on the performance of the robot in the initial episodes, a substantial reduction in the cost of the overall learning process, and in the convergence time to the (near-)optimal policy.