What's in a Query: Polarity-Aware Distribution-Based Fair Ranking

• URL: http://arxiv.org/abs/2502.11429v1
• Date: Mon, 17 Feb 2025 04:38:36 GMT
• Title: What's in a Query: Polarity-Aware Distribution-Based Fair Ranking
• Authors: Aparna Balagopalan, Kai Wang, Olawale Salaudeen, Asia Biega, Marzyeh Ghassemi,
• Abstract summary: We propose new measures for attention distribution-based fairness in ranking. We prove that group fairness is upper-bounded by individual fairness. We find that prior research in amortized fair ranking ignores critical information about queries.
• Score: 15.602253525743798
• License:
• Abstract: Machine learning-driven rankings, where individuals (or items) are ranked in response to a query, mediate search exposure or attention in a variety of safety-critical settings. Thus, it is important to ensure that such rankings are fair. Under the goal of equal opportunity, attention allocated to an individual on a ranking interface should be proportional to their relevance across search queries. In this work, we examine amortized fair ranking -- where relevance and attention are cumulated over a sequence of user queries to make fair ranking more feasible in practice. Unlike prior methods that operate on expected amortized attention for each individual, we define new divergence-based measures for attention distribution-based fairness in ranking (DistFaiR), characterizing unfairness as the divergence between the distribution of attention and relevance corresponding to an individual over time. This allows us to propose new definitions of unfairness, which are more reliable at test time. Second, we prove that group fairness is upper-bounded by individual fairness under this definition for a useful class of divergence measures, and experimentally show that maximizing individual fairness through an integer linear programming-based optimization is often beneficial to group fairness. Lastly, we find that prior research in amortized fair ranking ignores critical information about queries, potentially leading to a fairwashing risk in practice by making rankings appear more fair than they actually are.

Related papers

• Stability and Multigroup Fairness in Ranking with Uncertain Predictions [61.76378420347408]
  Our work considers ranking functions: maps from individual predictions for a classification task to distributions over rankings. We focus on two aspects of ranking functions: stability to perturbations in predictions and fairness towards both individuals and subgroups. Our work demonstrates that uncertainty aware rankings naturally interpolate between group and individual level fairness guarantees.
  arXiv  Detail & Related papers  (2024-02-14T17:17:05Z)
• Fairness in Ranking under Disparate Uncertainty [24.401219403555814]
  We argue that ranking can introduce unfairness if the uncertainty of the underlying relevance model differs between groups of options. We propose Equal-Opportunity Ranking (EOR) as a new fairness criterion for ranking. We show that EOR corresponds to a group-wise fair lottery among the relevant options even in the presence of disparate uncertainty.
  arXiv  Detail & Related papers  (2023-09-04T13:49:48Z)
• Bipartite Ranking Fairness through a Model Agnostic Ordering Adjustment [54.179859639868646]
  We propose a model agnostic post-processing framework xOrder for achieving fairness in bipartite ranking. xOrder is compatible with various classification models and ranking fairness metrics, including supervised and unsupervised fairness metrics. We evaluate our proposed algorithm on four benchmark data sets and two real-world patient electronic health record repositories.
  arXiv  Detail & Related papers  (2023-07-27T07:42:44Z)
• Sampling Individually-Fair Rankings that are Always Group Fair [9.333939443470944]
  A fair ranking task asks to rank a set of items to maximize utility subject to satisfying group-fairness constraints. Recent works identify uncertainty in the utilities of items as a primary cause of unfairness. We give an efficient algorithm that samples rankings from an individually-fair distribution while ensuring that every output ranking is group fair.
  arXiv  Detail & Related papers  (2023-06-21T01:26:34Z)
• The Role of Relevance in Fair Ranking [1.5469452301122177]
  We argue that relevance scores should satisfy a set of desired criteria in order to guide fairness interventions. We then empirically show that not all of these criteria are met in a case study of relevance inferred from biased user click data. Our analyses and results surface the pressing need for new approaches to relevance collection and generation.
  arXiv  Detail & Related papers  (2023-05-09T16:58:23Z)
• Fairness in Matching under Uncertainty [78.39459690570531]
  algorithmic two-sided marketplaces have drawn attention to the issue of fairness in such settings. We axiomatize a notion of individual fairness in the two-sided marketplace setting which respects the uncertainty in the merits. We design a linear programming framework to find fair utility-maximizing distributions over allocations.
  arXiv  Detail & Related papers  (2023-02-08T00:30:32Z)
• Incentives for Item Duplication under Fair Ranking Policies [69.14168955766847]
  We study the behaviour of different fair ranking policies in the presence of duplicates. We find that fairness-aware ranking policies may conflict with diversity, due to their potential to incentivize duplication more than policies solely focused on relevance.
  arXiv  Detail & Related papers  (2021-10-29T11:11:15Z)
• MultiFair: Multi-Group Fairness in Machine Learning [52.24956510371455]
  We study multi-group fairness in machine learning (MultiFair) We propose a generic end-to-end algorithmic framework to solve it. Our proposed framework is generalizable to many different settings.
  arXiv  Detail & Related papers  (2021-05-24T02:30:22Z)
• Fairness Through Regularization for Learning to Rank [33.52974791836553]
  We show how to transfer numerous fairness notions from binary classification to a learning to rank context. Our formalism allows us to design a method for incorporating fairness objectives with provable generalization guarantees.
  arXiv  Detail & Related papers  (2021-02-11T13:29:08Z)
• Towards Model-Agnostic Post-Hoc Adjustment for Balancing Ranking Fairness and Algorithm Utility [54.179859639868646]
  Bipartite ranking aims to learn a scoring function that ranks positive individuals higher than negative ones from labeled data. There have been rising concerns on whether the learned scoring function can cause systematic disparity across different protected groups. We propose a model post-processing framework for balancing them in the bipartite ranking scenario.
  arXiv  Detail & Related papers  (2020-06-15T10:08:39Z)
• Fast Fair Regression via Efficient Approximations of Mutual Information [0.0]
  This paper introduces fast approximations of the independence, separation and sufficiency group fairness criteria for regression models. It uses such approximations as regularisers to enforce fairness within a regularised risk minimisation framework. Experiments in real-world datasets indicate that in spite of its superior computational efficiency our algorithm still displays state-of-the-art accuracy/fairness tradeoffs.
  arXiv  Detail & Related papers  (2020-02-14T08:50:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.