A Cooperative-Competitive Multi-Agent Framework for Auto-bidding in Online Advertising

• URL: http://arxiv.org/abs/2106.06224v1
• Date: Fri, 11 Jun 2021 08:07:14 GMT
• Title: A Cooperative-Competitive Multi-Agent Framework for Auto-bidding in Online Advertising
• Authors: Chao Wen, Miao Xu, Zhilin Zhang, Zhenzhe Zheng, Yuhui Wang, Xiangyu Liu, Yu Rong, Dong Xie, Xiaoyang Tan, Chuan Yu, Jian Xu, Fan Wu, Guihai Chen, Xiaoqiang Zhu
• Abstract summary: We propose a general Multi-Agent reinforcement learning framework for Auto-Bidding, namely MAAB, to learn the auto-bidding strategies. Our approach outperforms several baseline methods in terms of social welfare and guarantees the ad platform's revenue.
• Score: 53.636153252400945
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In online advertising, auto-bidding has become an essential tool for advertisers to optimize their preferred ad performance metrics by simply expressing the high-level campaign objectives and constraints. Previous works consider the design of auto-bidding agents from the single-agent view without modeling the mutual influence between agents. In this paper, we instead consider this problem from the perspective of a distributed multi-agent system, and propose a general Multi-Agent reinforcement learning framework for Auto-Bidding, namely MAAB, to learn the auto-bidding strategies. First, we investigate the competition and cooperation relation among auto-bidding agents, and propose temperature-regularized credit assignment for establishing a mixed cooperative-competitive paradigm. By carefully making a competition and cooperation trade-off among the agents, we can reach an equilibrium state that guarantees not only individual advertiser's utility but also the system performance (social welfare). Second, due to the observed collusion behaviors of bidding low prices underlying the cooperation, we further propose bar agents to set a personalized bidding bar for each agent, and then to alleviate the degradation of revenue. Third, to deploy MAAB to the large-scale advertising system with millions of advertisers, we propose a mean-field approach. By grouping advertisers with the same objective as a mean auto-bidding agent, the interactions among advertisers are greatly simplified, making it practical to train MAAB efficiently. Extensive experiments on the offline industrial dataset and Alibaba advertising platform demonstrate that our approach outperforms several baseline methods in terms of social welfare and guarantees the ad platform's revenue.

Related papers

• AgentVerse: Facilitating Multi-Agent Collaboration and Exploring Emergent Behaviors [93.38830440346783]
  We propose a multi-agent framework framework that can collaboratively adjust its composition as a greater-than-the-sum-of-its-parts system. Our experiments demonstrate that framework framework can effectively deploy multi-agent groups that outperform a single agent. In view of these behaviors, we discuss some possible strategies to leverage positive ones and mitigate negative ones for improving the collaborative potential of multi-agent groups.
  arXiv  Detail & Related papers  (2023-08-21T16:47:11Z)
• Bidding Strategies for Proportional Representation in Advertisement Campaigns [8.269283912626873]
  We show that equitable bidding may not result in equitable outcomes due to heterogeneous levels of competition for different types of individuals. We consider alterations that make no change to the platform mechanism and instead change the bidding strategies used by advertisers.
  arXiv  Detail & Related papers  (2023-05-22T23:29:05Z)
• Multi-agent Deep Covering Skill Discovery [50.812414209206054]
  We propose Multi-agent Deep Covering Option Discovery, which constructs the multi-agent options through minimizing the expected cover time of the multiple agents' joint state space. Also, we propose a novel framework to adopt the multi-agent options in the MARL process. We show that the proposed algorithm can effectively capture the agent interactions with the attention mechanism, successfully identify multi-agent options, and significantly outperforms prior works using single-agent options or no options.
  arXiv  Detail & Related papers  (2022-10-07T00:40:59Z)
• Functional Optimization Reinforcement Learning for Real-Time Bidding [14.5826735379053]
  Real-time bidding is the new paradigm of programmatic advertising. Existing approaches are struggling to provide a satisfactory solution for bidding optimization. This paper proposes a multi-agent reinforcement learning architecture for RTB with functional optimization.
  arXiv  Detail & Related papers  (2022-06-25T06:12:17Z)
• VFed-SSD: Towards Practical Vertical Federated Advertising [53.08038962443853]
  We propose a semi-supervised split distillation framework VFed-SSD to alleviate the two limitations. Specifically, we develop a self-supervised task MatchedPair Detection (MPD) to exploit the vertically partitioned unlabeled data. Our framework provides an efficient federation-enhanced solution for real-time display advertising with minimal deploying cost and significant performance lift.
  arXiv  Detail & Related papers  (2022-05-31T17:45:30Z)
• Bidding Agent Design in the LinkedIn Ad Marketplace [16.815498720115443]
  We establish a general optimization framework for the design of automated bidding agent in online marketplaces. As a result, the framework allows, for instance, the joint optimization of a group of ads across multiple platforms each running its own auction format. We share practical learnings of the deployed bidding system in the LinkedIn ad marketplace based on this framework.
  arXiv  Detail & Related papers  (2022-02-25T03:01:57Z)
• Multi-Agent Cooperative Bidding Games for Multi-Objective Optimization in e-Commercial Sponsored Search [26.117969395228503]
  We propose a novel multi-objective cooperative bid optimization formulation called Multi-Agent Cooperative bidding Games (MACG) A global objective to maximize the overall profit of all advertisements is added in order to encourage better cooperation and also to protect self-bidding advertisers. offline experiments and online A/B tests conducted on the Taobao platform indicate both single advertiser's objective and global profit have been significantly improved.
  arXiv  Detail & Related papers  (2021-06-08T03:18:28Z)
• We Know What You Want: An Advertising Strategy Recommender System for Online Advertising [26.261736843187045]
  We propose a recommender system for dynamic bidding strategy recommendation on display advertising platform. We use a neural network as the agent to predict the advertisers' demands based on their profile and historical adoption behaviors. Online evaluations show that the system can optimize the advertisers' advertising performance.
  arXiv  Detail & Related papers  (2021-05-25T17:06:59Z)
• A novel auction system for selecting advertisements in Real-Time bidding [68.8204255655161]
  Real-Time Bidding is a new Internet advertising system that has become very popular in recent years. We propose an alternative betting system with a new approach that not only considers the economic aspect but also other relevant factors for the functioning of the advertising system.
  arXiv  Detail & Related papers  (2020-10-22T18:36:41Z)
• Dynamic Knapsack Optimization Towards Efficient Multi-Channel Sequential Advertising [52.3825928886714]
  We formulate the sequential advertising strategy optimization as a dynamic knapsack problem. We propose a theoretically guaranteed bilevel optimization framework, which significantly reduces the solution space of the original optimization space. To improve the exploration efficiency of reinforcement learning, we also devise an effective action space reduction approach.
  arXiv  Detail & Related papers  (2020-06-29T18:50:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.