Refining Labeling Functions with Limited Labeled Data

• URL: http://arxiv.org/abs/2505.23470v2
• Date: Wed, 04 Jun 2025 04:14:44 GMT
• Title: Refining Labeling Functions with Limited Labeled Data
• Authors: Chenjie Li, Amir Gilad, Boris Glavic, Zhengjie Miao, Sudeepa Roy,
• Abstract summary: Programmatic weak supervision (PWS) significantly reduces human effort for labeling data by combining the outputs of user-provided labeling functions (LFs) on unlabeled datapoints.<n>We study the problem of fixing LFs based on a small set of labeled examples.<n>We develop novel techniques for repairing a set of LFs by minimally changing their results on the labeled examples.
• Score: 18.404750370538963
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Programmatic weak supervision (PWS) significantly reduces human effort for labeling data by combining the outputs of user-provided labeling functions (LFs) on unlabeled datapoints. However, the quality of the generated labels depends directly on the accuracy of the LFs. In this work, we study the problem of fixing LFs based on a small set of labeled examples. Towards this goal, we develop novel techniques for repairing a set of LFs by minimally changing their results on the labeled examples such that the fixed LFs ensure that (i) there is sufficient evidence for the correct label of each labeled datapoint and (ii) the accuracy of each repaired LF is sufficiently high. We model LFs as conditional rules which enables us to refine them, i.e., to selectively change their output for some inputs. We demonstrate experimentally that our system improves the quality of LFs based on surprisingly small sets of labeled datapoints.

Related papers

• Are LLMs Better than Reported? Detecting Label Errors and Mitigating Their Effect on Model Performance [21.926934384262594]
  Large language models (LLMs) offer new opportunities to enhance the annotation process. We compare expert, crowd-sourced, and our LLM-based annotations in terms of agreement, label quality, and efficiency. Our findings reveal a substantial number of label errors, which, when corrected, induce a significant upward shift in reported model performance.
  arXiv  Detail & Related papers  (2024-10-24T16:27:03Z)
• Generalized Semi-Supervised Learning via Self-Supervised Feature Adaptation [87.17768598044427]
  Traditional semi-supervised learning assumes that the feature distributions of labeled and unlabeled data are consistent. We propose Self-Supervised Feature Adaptation (SSFA), a generic framework for improving SSL performance when labeled and unlabeled data come from different distributions. Our proposed SSFA is applicable to various pseudo-label-based SSL learners and significantly improves performance in labeled, unlabeled, and even unseen distributions.
  arXiv  Detail & Related papers  (2024-05-31T03:13:45Z)
• Inaccurate Label Distribution Learning with Dependency Noise [52.08553913094809]
  We introduce the Dependent Noise-based Inaccurate Label Distribution Learning (DN-ILDL) framework to tackle the challenges posed by noise in label distribution learning. We show that DN-ILDL effectively addresses the ILDL problem and outperforms existing LDL methods.
  arXiv  Detail & Related papers  (2024-05-26T07:58:07Z)
• Deep Partial Multi-Label Learning with Graph Disambiguation [27.908565535292723]
  We propose a novel deep Partial multi-Label model with grAph-disambIguatioN (PLAIN) Specifically, we introduce the instance-level and label-level similarities to recover label confidences. At each training epoch, labels are propagated on the instance and label graphs to produce relatively accurate pseudo-labels.
  arXiv  Detail & Related papers  (2023-05-10T04:02:08Z)
• SoftMatch: Addressing the Quantity-Quality Trade-off in Semi-supervised Learning [101.86916775218403]
  This paper revisits the popular pseudo-labeling methods via a unified sample weighting formulation. We propose SoftMatch to overcome the trade-off by maintaining both high quantity and high quality of pseudo-labels during training. In experiments, SoftMatch shows substantial improvements across a wide variety of benchmarks, including image, text, and imbalanced classification.
  arXiv  Detail & Related papers  (2023-01-26T03:53:25Z)
• Ground Truth Inference for Weakly Supervised Entity Matching [76.6732856489872]
  We propose a simple but powerful labeling model for weak supervision tasks. We then tailor the labeling model specifically to the task of entity matching. We show that our labeling model results in a 9% higher F1 score on average than the best existing method.
  arXiv  Detail & Related papers  (2022-11-13T17:57:07Z)
• Sparse Conditional Hidden Markov Model for Weakly Supervised Named Entity Recognition [68.68300358332156]
  We propose the sparse conditional hidden Markov model (Sparse-CHMM) to evaluate noisy labeling functions. Sparse-CHMM is optimized through unsupervised learning with a three-stage training pipeline. It achieves a 3.01 average F1 score improvement on five comprehensive datasets.
  arXiv  Detail & Related papers  (2022-05-27T20:47:30Z)
• ULF: Unsupervised Labeling Function Correction using Cross-Validation for Weak Supervision [5.566060402907773]
  Weak supervision (WS) is a cost-effective alternative to manual data labeling. We introduce a new algorithm ULF for Unsupervised Labeling Function correction. ULF refines the allocation of LFs to classes by re-estimating this assignment on highly reliable cross-validated samples.
  arXiv  Detail & Related papers  (2022-04-14T10:29:01Z)
• Label Augmentation with Reinforced Labeling for Weak Supervision [0.1529342790344802]
  This paper proposes a new approach called reinforced labeling (RL) RL augments the LFs' outputs to cases not covered by LFs based on similarities among samples. Experiments on several domains (classification of YouTube comments, wine quality, and weather prediction) result in considerable gains.
  arXiv  Detail & Related papers  (2022-04-13T14:54:02Z)
• Learning to Robustly Aggregate Labeling Functions for Semi-supervised Data Programming [14.639568384768042]
  A critical bottleneck in supervised machine learning is the need for large amounts of labeled data. In this work, we propose an LF based reweighting framework ouralgo to solve these two critical limitations. Our algorithm learns a joint model on the (same) labeled dataset used for LF induction along with any unlabeled data in a semi-supervised manner.
  arXiv  Detail & Related papers  (2021-09-23T14:42:46Z)
• Self-Tuning for Data-Efficient Deep Learning [75.34320911480008]
  Self-Tuning is a novel approach to enable data-efficient deep learning. It unifies the exploration of labeled and unlabeled data and the transfer of a pre-trained model. It outperforms its SSL and TL counterparts on five tasks by sharp margins.
  arXiv  Detail & Related papers  (2021-02-25T14:56:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.