Transferable Environment Model With Disentangled Dynamics

Abstract

Foresight is a manifestation of human-level intelligence. In model-based reinforcement learning, obtaining a perfect environment model is essential and significant. Although recent research has achieved impressive advances in improving prediction accuracy, existing environment models are unable to generalize well across environments, mainly because appearance-independent dynamics learning completely relies on appearance-specific representation learning. In this paper, we propose a novel predictive model named transferable environment model (TEM) which disentangles state representation and dynamics. The disentanglement allows the model to deal with different observation distributions and meanwhile share a cross-environment latent dynamics. In a 3D visual platform, we show that our model has good generalization performance in target environments with very few data. Furthermore, the TEM is able to continually adapt to a sequence of target environments without forgetting the knowledge for previous environments. To the best of our knowledge, this paper is the first to endow a predictive model with the ability to work across multiple environments.

Keywords

• Environment model
• model-based reinforcement learning
• domain adaptation
• disentangled representation