Standardization of Weighted Ranking Correlation Coefficients

• URL: http://arxiv.org/abs/2504.08428v1
• Date: Fri, 11 Apr 2025 10:37:19 GMT
• Title: Standardization of Weighted Ranking Correlation Coefficients
• Authors: Pierangelo Lombardo,
• Abstract summary: A relevant problem in statistics is defining the correlation of two rankings of a list of items.<n>We propose a standardization function $g(x)$ that maps a correlation ranking coefficient $Gamma$ in a standard form $g(Gamma)$ that has zero expected value.
• Score: 0.06526824510982801
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A relevant problem in statistics is defining the correlation of two rankings of a list of items. Kendall's tau and Spearman's rho are two well established correlation coefficients, characterized by a symmetric form that ensures zero expected value between two pairs of rankings randomly chosen with uniform probability. However, in recent years, several weighted versions of the original Spearman and Kendall coefficients have emerged that take into account the greater importance of top ranks compared to low ranks, which is common in many contexts. The weighting schemes break the symmetry, causing a non-zero expected value between two random rankings. This issue is very relevant, as it undermines the concept of uncorrelation between rankings. In this paper, we address this problem by proposing a standardization function $g(x)$ that maps a correlation ranking coefficient $\Gamma$ in a standard form $g(\Gamma)$ that has zero expected value, while maintaining the relevant statistical properties of $\Gamma$.

Related papers

• A Causal Information-Flow Framework for Unbiased Learning-to-Rank [52.54102347581931]
  In web search and recommendation systems, user clicks are widely used to train ranking models.<n>We introduce a novel causal learning-based ranking framework that extends Unbiased Learning-to-Rank.<n>Our method consistently reduces measured bias leakage and improves ranking performance.
  arXiv  Detail & Related papers  (2026-01-09T07:19:35Z)
• What Is the Optimal Ranking Score Between Precision and Recall? We Can Always Find It and It Is Rarely $F_1$ [17.215680052668244]
  We establish that $F_$-induced rankings are meaningful and define a shortest path between precision- and recall-induced rankings.<n>We provide theoretical tools and a closed-form expression to find the optimal value for $$ for any distribution or set of performances.
  arXiv  Detail & Related papers  (2025-11-27T13:29:50Z)
• Semiparametric conformal prediction [79.6147286161434]
  We construct a conformal prediction set accounting for the joint correlation structure of the vector-valued non-conformity scores.<n>We flexibly estimate the joint cumulative distribution function (CDF) of the scores.<n>Our method yields desired coverage and competitive efficiency on a range of real-world regression problems.
  arXiv  Detail & Related papers  (2024-11-04T14:29:02Z)
• Stability and Multigroup Fairness in Ranking with Uncertain Predictions [61.76378420347408]
  Our work considers ranking functions: maps from individual predictions for a classification task to distributions over rankings. We focus on two aspects of ranking functions: stability to perturbations in predictions and fairness towards both individuals and subgroups. Our work demonstrates that uncertainty aware rankings naturally interpolate between group and individual level fairness guarantees.
  arXiv  Detail & Related papers  (2024-02-14T17:17:05Z)
• Found in the Middle: Permutation Self-Consistency Improves Listwise Ranking in Large Language Models [63.714662435555674]
  Large language models (LLMs) exhibit positional bias in how they use context. We propose permutation self-consistency, a form of self-consistency over ranking list outputs of black-box LLMs. Our approach improves scores from conventional inference by up to 7-18% for GPT-3.5 and 8-16% for LLaMA v2 (70B)
  arXiv  Detail & Related papers  (2023-10-11T17:59:02Z)
• Variance-Aware Regret Bounds for Stochastic Contextual Dueling Bandits [53.281230333364505]
  This paper studies the problem of contextual dueling bandits, where the binary comparison of dueling arms is generated from a generalized linear model (GLM) We propose a new SupLinUCB-type algorithm that enjoys computational efficiency and a variance-aware regret bound $tilde Obig(dsqrtsum_t=1Tsigma_t2 + dbig)$. Our regret bound naturally aligns with the intuitive expectation in scenarios where the comparison is deterministic, the algorithm only suffers from an $tilde O(d)$ regret.
  arXiv  Detail & Related papers  (2023-10-02T08:15:52Z)
• Weighting by Tying: A New Approach to Weighted Rank Correlation [3.2090220512332106]
  We propose a weighted rank correlation measure on the basis of fuzzy order relations. Our measure, called scaled gamma, is related to Goodman and Kruskal's gamma rank correlation.
  arXiv  Detail & Related papers  (2023-08-21T10:40:21Z)
• Uncertainty Quantification of MLE for Entity Ranking with Covariates [3.2839905453386162]
  This paper concerns with statistical estimation and inference for the ranking problems based on pairwise comparisons. We propose a novel model, Co-Assisted Ranking Estimation (CARE) model, that extends the well-known Bradley-Terry-Luce (BTL) model. We derive the maximum likelihood estimator of $alpha_i*_i=1n$ and $beta*$ under a sparse comparison graph. We validate our theoretical results through large-scale numerical studies and an application to the mutual fund stock holding dataset.
  arXiv  Detail & Related papers  (2022-12-20T02:28:27Z)
• Goodness of Fit Metrics for Multi-class Predictor [0.0]
  Several metrics are commonly used to measure fit goodness. A leading constraint at least in emphreal world multi-class problems is imbalanced data. We suggest generalizing Matthew's correlation coefficient into multi-dimensions.
  arXiv  Detail & Related papers  (2022-08-11T06:07:29Z)
• Statistical Depth Functions for Ranking Distributions: Definitions, Statistical Learning and Applications [3.7564482287844205]
  The concept of median/consensus has been widely investigated in order to provide a statistical summary of ranking data. It is the purpose of this paper to define analogs of quantiles, ranks and statistical procedures based on such quantities.
  arXiv  Detail & Related papers  (2022-01-20T10:30:56Z)
• Optimal Full Ranking from Pairwise Comparisons [16.852801934478475]
  The minimax rate of ranking exhibits a transition between an exponential rate and a rate depending on the magnitude of the signal-to-noise ratio of the problem. To achieve the minimax rate, we propose a divide-and-conquer ranking algorithm that first divides the $n$ players into groups of similar skills and then computes local MLE within each group.
  arXiv  Detail & Related papers  (2021-01-21T03:34:44Z)
• Sharp Statistical Guarantees for Adversarially Robust Gaussian Classification [54.22421582955454]
  We provide the first result of the optimal minimax guarantees for the excess risk for adversarially robust classification. Results are stated in terms of the Adversarial Signal-to-Noise Ratio (AdvSNR), which generalizes a similar notion for standard linear classification to the adversarial setting.
  arXiv  Detail & Related papers  (2020-06-29T21:06:52Z)
• SetRank: A Setwise Bayesian Approach for Collaborative Ranking from Implicit Feedback [50.13745601531148]
  We propose a novel setwise Bayesian approach for collaborative ranking, namely SetRank, to accommodate the characteristics of implicit feedback in recommender system. Specifically, SetRank aims at maximizing the posterior probability of novel setwise preference comparisons. We also present the theoretical analysis of SetRank to show that the bound of excess risk can be proportional to $sqrtM/N$.
  arXiv  Detail & Related papers  (2020-02-23T06:40:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.