Targeted multi-agent communication algorithm based on state control

Abstract

As an important mechanism in multi-agent interaction, communication can make agents form complex team relationships rather than constitute a simple set of multiple independent agents. However, the existing communication schemes can bring much timing redundancy and irrelevant messages, which seriously affects their practical application. To solve this problem, this paper proposes a targeted multi-agent communication algorithm based on state control (SCTC). The SCTC uses a gating mechanism based on state control to reduce the timing redundancy of communication between agents and determines the interaction relationship between agents and the importance weight of a communication message through a series connection of hard- and self-attention mechanisms, realizing targeted communication message processing. In addition, by minimizing the difference between the fusion message generated from a real communication message of each agent and a fusion message generated from the buffered message, the correctness of the final action choice of the agent is ensured. Our evaluation using a challenging set of StarCraft II benchmarks indicates that the SCTC can significantly improve the learning performance and reduce the communication overhead between agents, thus ensuring better cooperation between agents.

Keywords

• Multi-agent deep reinforcement learning
• State control
• Targeted interaction
• Communication mechanism