External Evaluation of Discrimination Mitigation Efforts in Meta's Ad Delivery

• URL: http://arxiv.org/abs/2506.16560v1
• Date: Thu, 19 Jun 2025 19:29:26 GMT
• Title: External Evaluation of Discrimination Mitigation Efforts in Meta's Ad Delivery
• Authors: Basileal Imana, Zeyu Shen, John Heidemann, Aleksandra Korolova,
• Abstract summary: We show that the Variance Reduction System (VRS) implemented by Meta does not meaningfully improve access to opportunities for individuals.<n>We then conduct experiments to evaluate VRS with real-world ads, and show that while VRS does reduce variance, it also raises advertiser costs.<n>We show our approach outperforms VRS by both increasing ad exposure for users from all groups and reducing cost to advertisers.
• Score: 47.418845064441605
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The 2022 settlement between Meta and the U.S. Department of Justice to resolve allegations of discriminatory advertising resulted is a first-of-its-kind change to Meta's ad delivery system aimed to address algorithmic discrimination in its housing ad delivery. In this work, we explore direct and indirect effects of both the settlement's choice of terms and the Variance Reduction System (VRS) implemented by Meta on the actual reduction in discrimination. We first show that the settlement terms allow for an implementation that does not meaningfully improve access to opportunities for individuals. The settlement measures impact of ad delivery in terms of impressions, instead of unique individuals reached by an ad; it allows the platform to level down access, reducing disparities by decreasing the overall access to opportunities; and it allows the platform to selectively apply VRS to only small advertisers. We then conduct experiments to evaluate VRS with real-world ads, and show that while VRS does reduce variance, it also raises advertiser costs (measured per-individuals-reached), therefore decreasing user exposure to opportunity ads for a given ad budget. VRS thus passes the cost of decreasing variance to advertisers. Finally, we explore an alternative approach to achieve the settlement goals, that is significantly more intuitive and transparent than VRS. We show our approach outperforms VRS by both increasing ad exposure for users from all groups and reducing cost to advertisers, thus demonstrating that the increase in cost to advertisers when implementing the settlement is not inevitable. Our methodologies use a black-box approach that relies on capabilities available to any regular advertiser, rather than on privileged access to data, allowing others to reproduce or extend our work.

Related papers

• Why am I Still Seeing This: Measuring the Effectiveness Of Ad Controls and Explanations in AI-Mediated Ad Targeting Systems [55.02903075972816]
  We evaluate the effectiveness of Meta's "See less" ad control and the actionability of ad targeting explanations following the shift to AI-mediated targeting. We find that utilizing the "See less" ad control for the topics we study does not significantly reduce the number of ads shown by Meta on these topics. We find that the majority of ad targeting explanations for local ads made no reference to location-specific targeting criteria.
  arXiv  Detail & Related papers  (2024-08-21T18:03:11Z)
• Auditing for Racial Discrimination in the Delivery of Education Ads [50.37313459134418]
  We propose a new third-party auditing method that can evaluate racial bias in the delivery of ads for education opportunities. We find evidence of racial discrimination in Meta's algorithmic delivery of ads for education opportunities, posing legal and ethical concerns.
  arXiv  Detail & Related papers  (2024-06-02T02:00:55Z)
• Ad-load Balancing via Off-policy Learning in a Content Marketplace [9.783697404304025]
  Ad-load balancing is a critical challenge in online advertising systems, particularly in the context of social media platforms. Traditional approaches to ad-load balancing rely on static allocation policies, which fail to adapt to changing user preferences and contextual factors. We present an approach that leverages off-policy learning and evaluation from logged bandit feedback.
  arXiv  Detail & Related papers  (2023-09-19T09:17:07Z)
• Online Ad Procurement in Non-stationary Autobidding Worlds [10.871587311621974]
  We introduce a primal-dual algorithm for online decision making with multi-dimension decision variables, bandit feedback and long-term uncertain constraints. We show that our algorithm achieves low regret in many worlds when procurement outcomes are generated through procedures that are adversarial, adversarially corrupted, periodic, and ergodic.
  arXiv  Detail & Related papers  (2023-07-10T00:41:08Z)
• Discrimination through Image Selection by Job Advertisers on Facebook [79.21648699199648]
  We propose and investigate the prevalence of a new means for discrimination in job advertising. It combines both targeting and delivery -- through the disproportionate representation or exclusion of people of certain demographics in job ad images. We use the Facebook Ad Library to demonstrate the prevalence of this practice.
  arXiv  Detail & Related papers  (2023-06-13T03:43:58Z)
• Towards Fairness in Personalized Ads Using Impression Variance Aware Reinforcement Learning [9.246089899723744]
  Variance Reduction System (VRS) for achieving more equitable outcomes in Meta's ads systems. We first define metrics to quantify fairness gaps in terms of ad impression variances. We then present the VRS for re-ranking ads in an impression variance-aware manner.
  arXiv  Detail & Related papers  (2023-06-05T22:38:21Z)
• Auditing for Discrimination in Algorithms Delivering Job Ads [70.02478301291264]
  We develop a new methodology for black-box auditing of algorithms for discrimination in the delivery of job advertisements. Our first contribution is to identify the distinction between skew in ad delivery due to protected categories such as gender or race. Second, we develop an auditing methodology that distinguishes between skew explainable by differences in qualifications from other factors. Third, we apply our proposed methodology to two prominent targeted advertising platforms for job ads: Facebook and LinkedIn.
  arXiv  Detail & Related papers  (2021-04-09T17:38:36Z)
• Dynamic Knapsack Optimization Towards Efficient Multi-Channel Sequential Advertising [52.3825928886714]
  We formulate the sequential advertising strategy optimization as a dynamic knapsack problem. We propose a theoretically guaranteed bilevel optimization framework, which significantly reduces the solution space of the original optimization space. To improve the exploration efficiency of reinforcement learning, we also devise an effective action space reduction approach.
  arXiv  Detail & Related papers  (2020-06-29T18:50:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.