RuPersonaChat: a dialog corpus for personalizing conversational agents

Abstract

Personalization is one of the keyways to improve the performance of conversational agents. It improves the quality of user interaction with a conversational agent and increases user satisfaction by increasing the consistency and specificity of responses. The dialogue with the agent becomes more consistent, the inconsistency of responses is reduced, and the responses become more specific and interesting. Training and testing personalized conversational agents requires specific datasets containing facts about a persona and texts of persona’s dialogues where replicas use those facts. There are several datasets in English and Chinese containing an average of five facts about a persona where the dialogues are composed by crowdsourcing users who repeatedly imitate different personas. This paper proposes a methodology for collecting an original dataset containing an extended set of facts about a persona and natural dialogues between personas. The new RuPersonaChat dataset is based on three different recording scenarios: an interview, a short conversation, and a long conversation. This is the first dataset for dialogue agent personalization collected which includes both natural dialogues and extended persona’s descriptions. Additionally, in the dataset, the persona’s replicas are annotated with the facts about the persona from which they are generated. The methodology for collecting an original corpus of test data proposed in this paper allows for testing language models for various tasks within the framework of personalized dialogue agent development. The collected dataset includes 139 dialogues and 2608 replicas. This dataset was used to test answer and question generation models and the best results were obtained using the Gpt3-large model (perplexity is equal to 15.7). The dataset can be used to test the personalized dialogue agents’ ability to talk about themselves to the interlocutor, to communicate with the interlocutor utilizing phatic speech and taking into account the extended context when communicating with the user.

Keywords

• data collection methodology
• dialog data
• conversational agents
• personalization
• question and answer generation