Bias-Aware Mislabeling Detection via Decoupled Confident Learning

• URL: http://arxiv.org/abs/2507.07216v2
• Date: Fri, 11 Jul 2025 16:34:30 GMT
• Title: Bias-Aware Mislabeling Detection via Decoupled Confident Learning
• Authors: Yunyi Li, Maria De-Arteaga, Maytal Saar-Tsechansky,
• Abstract summary: We propose Decoupled Confident Learning (DeCoLe) to detect mislabeled instances in datasets affected by label bias.<n>DeCoLe excels at bias aware mislabeling detection, consistently outperforming alternative approaches for label error detection.<n>Our work identifies and addresses the challenge of bias aware mislabeling detection and offers guidance on how DeCoLe can be integrated into organizational data management practices.
• Score: 12.45833130404355
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Reliable data is a cornerstone of modern organizational systems. A notable data integrity challenge stems from label bias, which refers to systematic errors in a label, a covariate that is central to a quantitative analysis, such that its quality differs across social groups. This type of bias has been conceptually and empirically explored and is widely recognized as a pressing issue across critical domains. However, effective methodologies for addressing it remain scarce. In this work, we propose Decoupled Confident Learning (DeCoLe), a principled machine learning based framework specifically designed to detect mislabeled instances in datasets affected by label bias, enabling bias aware mislabelling detection and facilitating data quality improvement. We theoretically justify the effectiveness of DeCoLe and evaluate its performance in the impactful context of hate speech detection, a domain where label bias is a well documented challenge. Empirical results demonstrate that DeCoLe excels at bias aware mislabeling detection, consistently outperforming alternative approaches for label error detection. Our work identifies and addresses the challenge of bias aware mislabeling detection and offers guidance on how DeCoLe can be integrated into organizational data management practices as a powerful tool to enhance data reliability.

Related papers

• Credible Teacher for Semi-Supervised Object Detection in Open Scene [106.25850299007674]
  In Open Scene Semi-Supervised Object Detection (O-SSOD), unlabeled data may contain unknown objects not observed in the labeled data. It is detrimental to the current methods that mainly rely on self-training, as more uncertainty leads to the lower localization and classification precision of pseudo labels. We propose Credible Teacher, an end-to-end framework to prevent uncertain pseudo labels from misleading the model.
  arXiv  Detail & Related papers  (2024-01-01T08:19:21Z)
• Mitigating Label Bias in Machine Learning: Fairness through Confident Learning [22.031325797588476]
  Discrimination can occur when the underlying unbiased labels are overwritten by an agent with potential bias. In this paper, we demonstrate that it is possible to eliminate bias by filtering the fairest instances within the framework of confident learning.
  arXiv  Detail & Related papers  (2023-12-14T08:55:38Z)
• Binary Classification with Confidence Difference [100.08818204756093]
  This paper delves into a novel weakly supervised binary classification problem called confidence-difference (ConfDiff) classification. We propose a risk-consistent approach to tackle this problem and show that the estimation error bound the optimal convergence rate. We also introduce a risk correction approach to mitigate overfitting problems, whose consistency and convergence rate are also proven.
  arXiv  Detail & Related papers  (2023-10-09T11:44:50Z)
• Mitigating Label Bias via Decoupled Confident Learning [14.001915875687862]
  Growing concerns regarding algorithmic fairness have led to a surge in methodologies to mitigate algorithmic bias. bias in labels is pervasive across important domains, including healthcare, hiring, and content moderation. We propose a pruning method -- Decoupled Confident Learning (DeCoLe) -- specifically designed to mitigate label bias.
  arXiv  Detail & Related papers  (2023-07-18T03:28:03Z)
• Systematic analysis of the impact of label noise correction on ML Fairness [0.0]
  We develop an empirical methodology to evaluate the effectiveness of label noise correction techniques in ensuring the fairness of models trained on biased datasets. Our results suggest that the Hybrid Label Noise Correction method achieves the best trade-off between predictive performance and fairness.
  arXiv  Detail & Related papers  (2023-06-28T08:08:14Z)
• Mitigating Label Noise through Data Ambiguation [9.51828574518325]
  Large models with high expressive power are prone to memorizing incorrect labels, thereby harming generalization performance. In this paper, we suggest to address the shortcomings of both methodologies by "ambiguating" the target information. More precisely, we leverage the framework of so-called superset learning to construct set-valued targets based on a confidence threshold.
  arXiv  Detail & Related papers  (2023-05-23T07:29:08Z)
• Class-Distribution-Aware Pseudo Labeling for Semi-Supervised Multi-Label Learning [97.88458953075205]
  Pseudo-labeling has emerged as a popular and effective approach for utilizing unlabeled data. This paper proposes a novel solution called Class-Aware Pseudo-Labeling (CAP) that performs pseudo-labeling in a class-aware manner.
  arXiv  Detail & Related papers  (2023-05-04T12:52:18Z)
• Dist-PU: Positive-Unlabeled Learning from a Label Distribution Perspective [89.5370481649529]
  We propose a label distribution perspective for PU learning in this paper. Motivated by this, we propose to pursue the label distribution consistency between predicted and ground-truth label distributions. Experiments on three benchmark datasets validate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2022-12-06T07:38:29Z)
• Automated Detection of Label Errors in Semantic Segmentation Datasets via Deep Learning and Uncertainty Quantification [5.279257531335345]
  We for the first time present a method for detecting label errors in semantic segmentation datasets with pixel-wise labels. Our approach is able to detect the vast majority of label errors while controlling the number of false label error detections.
  arXiv  Detail & Related papers  (2022-07-13T10:25:23Z)
• Debiased Pseudo Labeling in Self-Training [77.83549261035277]
  Deep neural networks achieve remarkable performances on a wide range of tasks with the aid of large-scale labeled datasets. To mitigate the requirement for labeled data, self-training is widely used in both academia and industry by pseudo labeling on readily-available unlabeled data. We propose Debiased, in which the generation and utilization of pseudo labels are decoupled by two independent heads.
  arXiv  Detail & Related papers  (2022-02-15T02:14:33Z)
• Exploiting Sample Uncertainty for Domain Adaptive Person Re-Identification [137.9939571408506]
  We estimate and exploit the credibility of the assigned pseudo-label of each sample to alleviate the influence of noisy labels. Our uncertainty-guided optimization brings significant improvement and achieves the state-of-the-art performance on benchmark datasets.
  arXiv  Detail & Related papers  (2020-12-16T04:09:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.