Adversarial Reinforcement Learning Against Statistic Inference on Agent Identity

Abstract

This paper considers an agent identity privacy problem in Markov decision process. There are two types of agents with different instantaneous control reward functions, e.g., two types of energy consumption activities in smart grid. An eavesdropper is assumed to intercept the observations of agent and make a statistic inference on the agent identity, which is privacy-sensitive and can be utilized by the eavesdropper to further make corresponding malicious attacks. With regard to the agent identity privacy problem, a privacy-preserving Markov decision process is formulated and a novel adversarial reinforcement learning algorithm is further proposed by exploiting the ideas of deep reinforcement learning and variational method to design the agent policies with the aim to optimally tradeoff improving cumulative control reward and preventing agent identity privacy leakage. Experiments in a modified OpenAI Gym environment show different training process patterns and justify the effectiveness of the proposed algorithm.

Keywords

• Kullback-Leibler divergence
• Markov decision process
• privacy-by-design
• variational method