Rankings from multimodal pairwise comparisons

• URL: http://arxiv.org/abs/2206.13580v1
• Date: Mon, 27 Jun 2022 18:50:51 GMT
• Title: Rankings from multimodal pairwise comparisons
• Authors: M. E. J. Newman
• Abstract summary: Given data on which competitors beat which others, the challenge is to rank the competitors from best to worst. Here we study the problem of computing rankings when there are multiple, potentially conflicting modes of comparison. We show that it is possible to compute a ranking in this situation and present a fast method for doing so, based on a combination of an expectation-maximization algorithm and a modified Bradley-Terry model.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The task of ranking individuals or teams, based on a set of comparisons between pairs, arises in various contexts, including sporting competitions and the analysis of dominance hierarchies among animals and humans. Given data on which competitors beat which others, the challenge is to rank the competitors from best to worst. Here we study the problem of computing rankings when there are multiple, potentially conflicting modes of comparison, such as multiple types of dominance behaviors among animals. We assume that we do not know a priori what information each behavior conveys about the ranking, or even whether they convey any information at all. Nonetheless we show that it is possible to compute a ranking in this situation and present a fast method for doing so, based on a combination of an expectation-maximization algorithm and a modified Bradley-Terry model. We give a selection of example applications to both animal and human competition.

Related papers

• Learning when to rank: Estimation of partial rankings from sparse, noisy comparisons [0.0]
  We develop a principled Bayesian methodology for learning partial rankings. Our framework is adaptable to any statistical ranking method. It gives a more parsimonious summary of the data than traditional ranking.
  arXiv  Detail & Related papers  (2025-01-05T11:04:30Z)
• Fair Pairs: Fairness-Aware Ranking Recovery from Pairwise Comparisons [2.056289813004423]
  We introduce the problem of fairness-aware ranking recovery from pairwise comparisons. We propose a group-conditioned accuracy measure which quantifies fairness of rankings recovered from pairwise comparisons.
  arXiv  Detail & Related papers  (2024-08-23T12:46:16Z)
• Luck, skill, and depth of competition in games and social hierarchies [0.6345523830122168]
  We find that social competition tends to be "deep," meaning it has a pronounced hierarchy with many distinct levels. In most cases there is little evidence of upset wins, beyond those already implied by the shallowness of the hierarchy.
  arXiv  Detail & Related papers  (2023-12-07T21:42:44Z)
• Competitions in AI -- Robustly Ranking Solvers Using Statistical Resampling [9.02080113915613]
  We show that rankings resulting from the standard interpretation of competition results can be very sensitive to even minor changes in the benchmark instance set used as the basis for assessment. We introduce a novel approach to statistically meaningful analysis of competition results based on resampling performance data. Our approach produces confidence intervals of competition scores as well as statistically robust solver rankings with bounded error.
  arXiv  Detail & Related papers  (2023-08-09T16:47:04Z)
• Bipartite Ranking Fairness through a Model Agnostic Ordering Adjustment [54.179859639868646]
  We propose a model agnostic post-processing framework xOrder for achieving fairness in bipartite ranking. xOrder is compatible with various classification models and ranking fairness metrics, including supervised and unsupervised fairness metrics. We evaluate our proposed algorithm on four benchmark data sets and two real-world patient electronic health record repositories.
  arXiv  Detail & Related papers  (2023-07-27T07:42:44Z)
• A Tale of HodgeRank and Spectral Method: Target Attack Against Rank Aggregation Is the Fixed Point of Adversarial Game [153.74942025516853]
  The intrinsic vulnerability of the rank aggregation methods is not well studied in the literature. In this paper, we focus on the purposeful adversary who desires to designate the aggregated results by modifying the pairwise data. The effectiveness of the suggested target attack strategies is demonstrated by a series of toy simulations and several real-world data experiments.
  arXiv  Detail & Related papers  (2022-09-13T05:59:02Z)
• Integrating Rankings into Quantized Scores in Peer Review [61.27794774537103]
  In peer review, reviewers are usually asked to provide scores for the papers. To mitigate this issue, conferences have started to ask reviewers to additionally provide a ranking of the papers they have reviewed. There are no standard procedure for using this ranking information and Area Chairs may use it in different ways. We take a principled approach to integrate the ranking information into the scores.
  arXiv  Detail & Related papers  (2022-04-05T19:39:13Z)
• GNNRank: Learning Global Rankings from Pairwise Comparisons via Directed Graph Neural Networks [68.61934077627085]
  We introduce GNNRank, a modeling framework compatible with any GNN capable of learning digraph embeddings. We show that our methods attain competitive and often superior performance compared with existing approaches.
  arXiv  Detail & Related papers  (2022-02-01T04:19:50Z)
• Poisoning Attack against Estimating from Pairwise Comparisons [140.9033911097995]
  Attackers have strong motivation and incentives to manipulate the ranking list. Data poisoning attacks on pairwise ranking algorithms can be formalized as the dynamic and static games between the ranker and the attacker. We propose two efficient poisoning attack algorithms and establish the associated theoretical guarantees.
  arXiv  Detail & Related papers  (2021-07-05T08:16:01Z)
• Optimal Full Ranking from Pairwise Comparisons [16.852801934478475]
  The minimax rate of ranking exhibits a transition between an exponential rate and a rate depending on the magnitude of the signal-to-noise ratio of the problem. To achieve the minimax rate, we propose a divide-and-conquer ranking algorithm that first divides the $n$ players into groups of similar skills and then computes local MLE within each group.
  arXiv  Detail & Related papers  (2021-01-21T03:34:44Z)
• On the Problem of Underranking in Group-Fair Ranking [8.963918049835375]
  Bias in ranking systems can worsen social and economic inequalities, polarize opinions, and reinforce stereotypes. In this paper, we formulate the problem of underranking in group-fair rankings, which was not addressed in previous work. We give a fair ranking algorithm that takes any given ranking and outputs another ranking with simultaneous underranking and group fairness guarantees.
  arXiv  Detail & Related papers  (2020-09-24T14:56:10Z)
• Towards Model-Agnostic Post-Hoc Adjustment for Balancing Ranking Fairness and Algorithm Utility [54.179859639868646]
  Bipartite ranking aims to learn a scoring function that ranks positive individuals higher than negative ones from labeled data. There have been rising concerns on whether the learned scoring function can cause systematic disparity across different protected groups. We propose a model post-processing framework for balancing them in the bipartite ranking scenario.
  arXiv  Detail & Related papers  (2020-06-15T10:08:39Z)
• Ranking a set of objects: a graph based least-square approach [70.7866286425868]
  We consider the problem of ranking $N$ objects starting from a set of noisy pairwise comparisons provided by a crowd of equal workers. We propose a class of non-adaptive ranking algorithms that rely on a least-squares intrinsic optimization criterion for the estimation of qualities.
  arXiv  Detail & Related papers  (2020-02-26T16:19:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.