Concentration Inequalities for Two-Sample Rank Processes with Application to Bipartite Ranking

• URL: http://arxiv.org/abs/2104.02943v1
• Date: Wed, 7 Apr 2021 06:31:06 GMT
• Title: Concentration Inequalities for Two-Sample Rank Processes with Application to Bipartite Ranking
• Abstract summary: The ROC curve is the gold standard for measuring the performance of a test/scoring statistic. Most practical performance measures used in scoring/ranking applications can be viewed as summaries of the ROC curve.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The ROC curve is the gold standard for measuring the performance of a test/scoring statistic regarding its capacity to discriminate between two statistical populations in a wide variety of applications, ranging from anomaly detection in signal processing to information retrieval, through medical diagnosis. Most practical performance measures used in scoring/ranking applications such as the AUC, the local AUC, the p-norm push, the DCG and others, can be viewed as summaries of the ROC curve. In this paper, the fact that most of these empirical criteria can be expressed as two-sample linear rank statistics is highlighted and concentration inequalities for collections of such random variables, referred to as two-sample rank processes here, are proved, when indexed by VC classes of scoring functions. Based on these nonasymptotic bounds, the generalization capacity of empirical maximizers of a wide class of ranking performance criteria is next investigated from a theoretical perspective. It is also supported by empirical evidence through convincing numerical experiments.

Related papers

• Statistical Verification of Linear Classifiers [76.95660509846216]
  We propose a homogeneity test closely related to the concept of linear separability between two samples. We focus on establishing upper bounds for the test's emphp-value when applied to two-dimensional samples.
  arXiv  Detail & Related papers  (2025-01-24T11:56:45Z)
• Generalized coarsened confounding for causal effects: a large-sample framework [3.9483979013722683]
  We consider a class of generalized coarsened procedures for confounding.<n>We propose two new algorithms for generalized coarsened confounding.<n>A bias correction technique is proposed, and we apply the proposed methodology to data from two well-known observational studies.
  arXiv  Detail & Related papers  (2025-01-06T16:47:24Z)
• Two Is Better Than One: Aligned Representation Pairs for Anomaly Detection [56.57122939745213]
  Anomaly detection focuses on identifying samples that deviate from the norm.<n>Recent self-supervised methods have successfully learned such representations by employing prior knowledge about anomalies to create synthetic outliers during training.<n>We address this limitation with our new approach Con$$, which leverages prior knowledge about symmetries in normal samples to observe the data in different contexts.
  arXiv  Detail & Related papers  (2024-05-29T07:59:06Z)
• Kernel Density Estimation for Multiclass Quantification [52.419589623702336]
  Quantification is the supervised machine learning task concerned with obtaining accurate predictors of class prevalence. The distribution-matching (DM) approaches represent one of the most important families among the quantification methods that have been proposed in the literature so far. We propose a new representation mechanism based on multivariate densities that we model via kernel density estimation (KDE)
  arXiv  Detail & Related papers  (2023-12-31T13:19:27Z)
• Analysis of Diagnostics (Part I): Prevalence, Uncertainty Quantification, and Machine Learning [0.0]
  This manuscript is the first in a two-part series that studies deeper connections between classification theory and prevalence. We propose a numerical, homotopy algorithm that estimates the $Bstar (q)$ by minimizing a prevalence-weighted empirical error. We validate our methods in the context of synthetic data and a research-use-only SARS-CoV-2 enzyme-linked immunosorbent (ELISA) assay.
  arXiv  Detail & Related papers  (2023-08-30T13:26:49Z)
• Spectral Ranking Inferences based on General Multiway Comparisons [7.222667862159246]
  We show that a two-step spectral method can achieve the same vanilla efficiency as the Maximum Likelihood Estor. It is noteworthy that this is the first time effective two-sample rank testing methods have been proposed.
  arXiv  Detail & Related papers  (2023-08-05T16:31:32Z)
• Learning disentangled representations for explainable chest X-ray classification using Dirichlet VAEs [68.73427163074015]
  This study explores the use of the Dirichlet Variational Autoencoder (DirVAE) for learning disentangled latent representations of chest X-ray (CXR) images. The predictive capacity of multi-modal latent representations learned by DirVAE models is investigated through implementation of an auxiliary multi-label classification task.
  arXiv  Detail & Related papers  (2023-02-06T18:10:08Z)
• Parametric Classification for Generalized Category Discovery: A Baseline Study [70.73212959385387]
  Generalized Category Discovery (GCD) aims to discover novel categories in unlabelled datasets using knowledge learned from labelled samples. We investigate the failure of parametric classifiers, verify the effectiveness of previous design choices when high-quality supervision is available, and identify unreliable pseudo-labels as a key problem. We propose a simple yet effective parametric classification method that benefits from entropy regularisation, achieves state-of-the-art performance on multiple GCD benchmarks and shows strong robustness to unknown class numbers.
  arXiv  Detail & Related papers  (2022-11-21T18:47:11Z)
• Hierarchical Semi-Supervised Contrastive Learning for Contamination-Resistant Anomaly Detection [81.07346419422605]
  Anomaly detection aims at identifying deviant samples from the normal data distribution. Contrastive learning has provided a successful way to sample representation that enables effective discrimination on anomalies. We propose a novel hierarchical semi-supervised contrastive learning framework, for contamination-resistant anomaly detection.
  arXiv  Detail & Related papers  (2022-07-24T18:49:26Z)
• Understanding the impact of class imbalance on the performance of chest x-ray image classifiers [13.593336529221613]
  This work aims to understand the impact of class imbalance on the performance of chest x-ray classifiers. We propose the adoption of two alternative metrics, the precision-recall curve and the Balanced Brier score, which better reflect the performance of the system.
  arXiv  Detail & Related papers  (2021-12-23T20:57:47Z)
• Learning to Rank Anomalies: Scalar Performance Criteria and Maximization of Two-Sample Rank Statistics [0.0]
  We propose a data-driven scoring function defined on the feature space which reflects the degree of abnormality of the observations. This scoring function is learnt through a well-designed binary classification problem. We illustrate our methodology with preliminary encouraging numerical experiments.
  arXiv  Detail & Related papers  (2021-09-20T14:45:56Z)
• Towards Model-Agnostic Post-Hoc Adjustment for Balancing Ranking Fairness and Algorithm Utility [54.179859639868646]
  Bipartite ranking aims to learn a scoring function that ranks positive individuals higher than negative ones from labeled data. There have been rising concerns on whether the learned scoring function can cause systematic disparity across different protected groups. We propose a model post-processing framework for balancing them in the bipartite ranking scenario.
  arXiv  Detail & Related papers  (2020-06-15T10:08:39Z)
• Learning Fair Scoring Functions: Bipartite Ranking under ROC-based Fairness Constraints [2.0263791972068628]
  We study fairness for the problem of learning scoring functions from binary labeled data. We introduce general families of fairness definitions based on the AUC and on ROC curves. We establish bounds generalization for scoring functions learned under such constraints.
  arXiv  Detail & Related papers  (2020-02-19T13:17:39Z)
• A Survey on Causal Inference [64.45536158710014]
  Causal inference is a critical research topic across many domains, such as statistics, computer science, education, public policy and economics. Various causal effect estimation methods for observational data have sprung up.
  arXiv  Detail & Related papers  (2020-02-05T21:35:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.