Detecting labeling bias using influence functions

• URL: http://arxiv.org/abs/2602.19130v1
• Date: Sun, 22 Feb 2026 11:20:35 GMT
• Title: Detecting labeling bias using influence functions
• Authors: Frida Jørgensen, Nina Weng, Siavash Bigdeli,
• Abstract summary: influence functions can be used to detect labeling bias.<n>We develop a sample valuation pipeline and test it first on the MNIST dataset.<n>On CheXpert, mislabeled samples consistently exhibit higher influence scores.
• Score: 1.9276091419970947
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Labeling bias arises during data collection due to resource limitations or unconscious bias, leading to unequal label error rates across subgroups or misrepresentation of subgroup prevalence. Most fairness constraints assume training labels reflect the true distribution, rendering them ineffective when labeling bias is present; leaving a challenging question, that \textit{how can we detect such labeling bias?} In this work, we investigate whether influence functions can be used to detect labeling bias. Influence functions estimate how much each training sample affects a model's predictions by leveraging the gradient and Hessian of the loss function -- when labeling errors occur, influence functions can identify wrongly labeled samples in the training set, revealing the underlying failure mode. We develop a sample valuation pipeline and test it first on the MNIST dataset, then scaled to the more complex CheXpert medical imaging dataset. To examine label noise, we introduced controlled errors by flipping 20\% of the labels for one class in the dataset. Using a diagonal Hessian approximation, we demonstrated promising results, successfully detecting nearly 90\% of mislabeled samples in MNIST. On CheXpert, mislabeled samples consistently exhibit higher influence scores. These results highlight the potential of influence functions for identifying label errors.

Related papers

• Enhanced Sample Selection with Confidence Tracking: Identifying Correctly Labeled yet Hard-to-Learn Samples in Noisy Data [18.111971239860836]
  We propose a novel sample selection method for image classification in the presence of noisy labels.<n>Our goal is to accurately distinguish correctly labeled yet hard-to-learn samples from mislabeled ones.<n>Our method functions as a plug-and-play component that can be seamlessly integrated into existing sample selection techniques.
  arXiv  Detail & Related papers  (2025-04-24T12:07:14Z)
• Label Distribution Learning with Biased Annotations by Learning Multi-Label Representation [120.97262070068224]
  Multi-label learning (MLL) has gained attention for its ability to represent real-world data.<n>Label Distribution Learning (LDL) faces challenges in collecting accurate label distributions.
  arXiv  Detail & Related papers  (2025-02-03T09:04:03Z)
• From Biased Selective Labels to Pseudo-Labels: An Expectation-Maximization Framework for Learning from Biased Decisions [9.440055827786596]
  We study a clinically-inspired selective label problem called disparate censorship. Disparate Censorship Expectation-Maximization (DCEM) is an algorithm for learning in the presence of such censorship.
  arXiv  Detail & Related papers  (2024-06-27T03:33:38Z)
• Soft Curriculum for Learning Conditional GANs with Noisy-Labeled and Uncurated Unlabeled Data [70.25049762295193]
  We introduce a novel conditional image generation framework that accepts noisy-labeled and uncurated data during training. We propose soft curriculum learning, which assigns instance-wise weights for adversarial training while assigning new labels for unlabeled data. Our experiments show that our approach outperforms existing semi-supervised and label-noise robust methods in terms of both quantitative and qualitative performance.
  arXiv  Detail & Related papers  (2023-07-17T08:31:59Z)
• Dist-PU: Positive-Unlabeled Learning from a Label Distribution Perspective [89.5370481649529]
  We propose a label distribution perspective for PU learning in this paper. Motivated by this, we propose to pursue the label distribution consistency between predicted and ground-truth label distributions. Experiments on three benchmark datasets validate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2022-12-06T07:38:29Z)
• How Does Pseudo-Labeling Affect the Generalization Error of the Semi-Supervised Gibbs Algorithm? [73.80001705134147]
  We provide an exact characterization of the expected generalization error (gen-error) for semi-supervised learning (SSL) with pseudo-labeling via the Gibbs algorithm. The gen-error is expressed in terms of the symmetrized KL information between the output hypothesis, the pseudo-labeled dataset, and the labeled dataset.
  arXiv  Detail & Related papers  (2022-10-15T04:11:56Z)
• SLA$^2$P: Self-supervised Anomaly Detection with Adversarial Perturbation [77.71161225100927]
  Anomaly detection is a fundamental yet challenging problem in machine learning. We propose a novel and powerful framework, dubbed as SLA$2$P, for unsupervised anomaly detection.
  arXiv  Detail & Related papers  (2021-11-25T03:53:43Z)
• Assessing the Quality of the Datasets by Identifying Mislabeled Samples [14.881597737762316]
  We propose a novel statistic -- noise score -- as a measure for the quality of each data point to identify mislabeled samples. In our work, we use the representations derived by the inference network of data quality supervised variational autoencoder (AQUAVS) We validate our proposed statistic through experimentation by corrupting MNIST, FashionMNIST, and CIFAR10/100 datasets.
  arXiv  Detail & Related papers  (2021-09-10T17:14:09Z)
• Disentangling Sampling and Labeling Bias for Learning in Large-Output Spaces [64.23172847182109]
  We show that different negative sampling schemes implicitly trade-off performance on dominant versus rare labels. We provide a unified means to explicitly tackle both sampling bias, arising from working with a subset of all labels, and labeling bias, which is inherent to the data due to label imbalance.
  arXiv  Detail & Related papers  (2021-05-12T15:40:13Z)
• Fair Classification with Group-Dependent Label Noise [6.324366770332667]
  This work examines how to train fair classifiers in settings where training labels are corrupted with random noise. We show that naively imposing parity constraints on demographic disparity measures, without accounting for heterogeneous and group-dependent error rates, can decrease both the accuracy and the fairness of the resulting classifier.
  arXiv  Detail & Related papers  (2020-10-31T22:35:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.