Real-World Robot Control Based on Contrastive Deep Active Inference With Demonstrations

Abstract

Despite significant advances in robotics and deep learning, the ability of robots to perceive and act remain far below that of humans. To bridge this gap, we utilize active inference, a framework based on the free-energy principle that accounts for various human brain functions. Despite the utility of active inference, most previous studies have focused on simulated or real-world environments with low-dimensional observations. Therefore, direct applications in real-world robotics with high-dimensional observation and action spaces remain a challenge. To address this issue, we combine the active inference framework with contrastive learning and develop an extension by introducing learning from demonstration. This framework comprises a world model, an action model, and an expected free-energy model. The world model ensures an adequate perception of the environment by minimizing contrastive variational free energy. Meanwhile, the expected free-energy model estimates the contrastive expected free energy, while the action model generates adaptive actions by learning from demonstrations, with the aim of minimizing the estimates provided by the expected free-energy model. We show the effectiveness and robustness of the proposed framework through image-based robotic reaching tasks in both learned and unlearned real-world environments. As a result, while the baselines achieved a success rate of up to approximately 71%, the proposed framework achieved a success rate of around 96%. Moreover, the proposed framework realized performance improvements by leveraging estimated expected free energy for action selection. These results highlight the utility of active inference with contrastive learning and learning from demonstration for real-world robot control.

Keywords

• Active inference
• robot learning
• learning from demonstration
• world model
• free energy principle
• contrastive learning