A Multi-Channel Advertising Budget Allocation Using Reinforcement Learning and an Improved Differential Evolution Algorithm

Abstract

Budget allocation across multiple advertising channels involves periodically dividing a fixed total budget among various channels. Yet, the challenge of making sequential decisions to optimize long-term benefits rather than short-term gains is often overlooked. Additionally, more apparent connections must be made between actions taken on one advertising channel and the outcomes on others. Furthermore, budget limitations narrow down the range of potential optimal strategies that can be pursued. In response to these challenges, this study unveils a pioneering multi-channel advertising budget allocation approach that leverages a reinforcement learning (RL) Q-learning framework enriched with an advanced Differential Evolution (DE) algorithm to refine the Q-learning methodology. The RL element makes informed sequential decisions, adeptly adjusting strategies to favor long-term rewards by assimilating environmental feedback. Complementing this, the enhanced DE algorithm introduces an inventive clustering-based mutation technique, exploiting key groupings within the DE population to generate novel and practical solutions. The model is further bolstered by a discretization tactic aimed at simplifying the model by streamlining costs. The proposed methodology is rigorously validated using two extensive datasets: the Chinese Internet Company Advertising Dataset (CICAD) and CRITEO-UPLIFT v2, employing metrics like Area Under the Cost Curve (AUCC) and Expected Outcome Metric (EOM) as measures of performance. The empirical results affirm the superiority of the model, showcasing its exceptional performance with significant scores (AUCC =0.750 and EOM =0.736 for CICAD; AUCC =0.813 and EOM =0.829 for CRITEO-UPLIFT v2), thereby illustrating the model’s proficiency in navigating the multifaceted challenges associated with multi-channel budget allocation and establishing a new benchmark in the field.

Keywords

• Budget allocation with constraint
• marketing
• social networks
• reinforcement learning
• differential evolution