Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization

Abstract

Multi-agent deep reinforcement learning based on value factorization is one of many multi-agent deep reinforcement learning algorithms,and it is also a research hotspot in the field of multi-agent deep reinforcement learning.Under some constraints,the joint action value function of multi-agent system is factorized into a certain combination of individual action value function,which is able to effectively solve the problems of environment instability and exponential explosion of action space in multi-agent system.Firstly,this paper explains why value function factorization should be carried out and introduces the basic theory of multi-agent deep reinforcement learning.Secondly,according to whether to introduce other mechanisms and the diffe-rence of introduced mechanism,multi-agent deep reinforcement learning(MADRL)algorithm based on value factorization is divi-ded into three categories:simple factorization type,based on the individual-global-max(IGM)principle and based on attention mechanism.Then,according to the classifications,this paper emphatically introduces several typical algorithms and compares and analyzes their strengths and weaknesses.Finally,it briefly describes the application and development prospect of these algorithms.

Keywords

• factorization of value function|madrl|attention mechanism|principle of igm