Structured Exploration and Exploitation of Label Functions for Automated Data Annotation

• URL: http://arxiv.org/abs/2604.08578v1
• Date: Sat, 28 Mar 2026 04:19:33 GMT
• Title: Structured Exploration and Exploitation of Label Functions for Automated Data Annotation
• Abstract summary: Programmatic labeling uses label functions (LFs), i.e., rules that automatically generate weak labels for training datasets.<n>In this paper, we introduce EXPONA, an automated framework for programmatic labeling that balances diversity and reliability.<n>Experiments show that EXPONA consistently outperformed state-of-the-art automated LF generation methods.
• Score: 3.780303340354419
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: High-quality labeled data is critical for training reliable machine learning and deep learning models, yet manual annotation remains costly and error-prone. Programmatic labeling addresses this challenge by using label functions (LFs), i.e., heuristic rules that automatically generate weak labels for training datasets. However, existing automated LF generation methods either rely on large language models (LLMs) to synthesize surface-level heuristics or employ model-based synthesis over hand-crafted primitives. These approaches often result in limited coverage and unreliable label quality. In this paper, we introduce EXPONA, an automated framework for programmatic labeling that formulates LF generation as a principled process balancing diversity and reliability. EXPONA systematically explores multi-level LFs, spanning surface, structural, and semantic perspectives. EXPONA further applies reliability-aware mechanisms to suppress noisy or redundant heuristics while preserving complementary signals. To evaluate EXPONA, we conducted extensive experiments on eleven classification datasets across diverse domains. Experimental results show that EXPONA consistently outperformed state-of-the-art automated LF generation methods. Specifically, EXPONA achieved nearly complete label coverage (up to 98.9%), improved weak label quality by up to 87%, and yielded downstream performance gains of up to 46% in weighted F1. These results indicate that EXPONA's combination of multi-level LF exploration and reliability-aware filtering enabled more consistent label quality and downstream performance across diverse tasks by balancing coverage and precision in the generated LF set.

Related papers

• Trustworthy Federated Label Distribution Learning under Annotation Quality Disparity [70.60312607268871]
  FedQual is a quality-aware Fed-LDL framework with two mechanisms.<n>We construct four new Fed-LDL benchmarks with controlled annotation quality disparity.
  arXiv  Detail & Related papers  (2026-05-06T12:20:20Z)
• Self-Improving Tabular Language Models via Iterative Group Alignment [76.43372236405664]
  We introduce TabGRAA, the first self-improving framework for data generation via automated feedback.<n>It partitions newly generated samples into high- and low-quality groups, then prioritizes a group-relative advantage objective.<n>Experiments show TabGRAA outperforms existing methods in fidelity, utility, and privacy.
  arXiv  Detail & Related papers  (2026-04-21T01:29:52Z)
• Refining Labeling Functions with Limited Labeled Data [18.404750370538963]
  Programmatic weak supervision (PWS) significantly reduces human effort for labeling data by combining the outputs of user-provided labeling functions (LFs) on unlabeled datapoints.<n>We study the problem of fixing LFs based on a small set of labeled examples.<n>We develop novel techniques for repairing a set of LFs by minimally changing their results on the labeled examples.
  arXiv  Detail & Related papers  (2025-05-29T14:26:11Z)
• Synthetic Code Surgery: Repairing Bugs and Vulnerabilities with LLMs and Synthetic Data [0.0]
  This paper presents a novel methodology for enhancing Automated Program Repair (APR) through synthetic data generation utilizing Large Language Models (LLMs)<n>The proposed approach addresses this limitation through a two-phase process: a synthetic sample generation followed by a rigorous quality assessment.<n> Experimental evaluation on the VulRepair test set dataset showed statistically significant improvements in Perfect Prediction rates.
  arXiv  Detail & Related papers  (2025-05-12T09:14:20Z)
• Co-training for Low Resource Scientific Natural Language Inference [65.37685198688538]
  We propose a novel co-training method that assigns weights based on the training dynamics of the classifiers to the distantly supervised labels. By assigning importance weights instead of filtering out examples based on an arbitrary threshold on the predicted confidence, we maximize the usage of automatically labeled data. The proposed method obtains an improvement of 1.5% in Macro F1 over the distant supervision baseline, and substantial improvements over several other strong SSL baselines.
  arXiv  Detail & Related papers  (2024-06-20T18:35:47Z)
• Online Feature Updates Improve Online (Generalized) Label Shift Adaptation [51.328801874640675]
  Our novel method, Online Label Shift adaptation with Online Feature Updates (OLS-OFU), leverages self-supervised learning to refine the feature extraction process. By carefully designing the algorithm, OLS-OFU maintains the similar online regret convergence to the results in the literature while taking the improved features into account.
  arXiv  Detail & Related papers  (2024-02-05T22:03:25Z)
• Leveraging Instance Features for Label Aggregation in Programmatic Weak Supervision [75.1860418333995]
  Programmatic Weak Supervision (PWS) has emerged as a widespread paradigm to synthesize training labels efficiently. The core component of PWS is the label model, which infers true labels by aggregating the outputs of multiple noisy supervision sources as labeling functions. Existing statistical label models typically rely only on the outputs of LF, ignoring the instance features when modeling the underlying generative process.
  arXiv  Detail & Related papers  (2022-10-06T07:28:53Z)
• Sparse Conditional Hidden Markov Model for Weakly Supervised Named Entity Recognition [68.68300358332156]
  We propose the sparse conditional hidden Markov model (Sparse-CHMM) to evaluate noisy labeling functions. Sparse-CHMM is optimized through unsupervised learning with a three-stage training pipeline. It achieves a 3.01 average F1 score improvement on five comprehensive datasets.
  arXiv  Detail & Related papers  (2022-05-27T20:47:30Z)
• ULF: Unsupervised Labeling Function Correction using Cross-Validation for Weak Supervision [5.566060402907773]
  Weak supervision (WS) is a cost-effective alternative to manual data labeling. We introduce a new algorithm ULF for Unsupervised Labeling Function correction. ULF refines the allocation of LFs to classes by re-estimating this assignment on highly reliable cross-validated samples.
  arXiv  Detail & Related papers  (2022-04-14T10:29:01Z)
• Label Augmentation with Reinforced Labeling for Weak Supervision [0.1529342790344802]
  This paper proposes a new approach called reinforced labeling (RL) RL augments the LFs' outputs to cases not covered by LFs based on similarities among samples. Experiments on several domains (classification of YouTube comments, wine quality, and weather prediction) result in considerable gains.
  arXiv  Detail & Related papers  (2022-04-13T14:54:02Z)
• Learning to Robustly Aggregate Labeling Functions for Semi-supervised Data Programming [14.639568384768042]
  A critical bottleneck in supervised machine learning is the need for large amounts of labeled data. In this work, we propose an LF based reweighting framework ouralgo to solve these two critical limitations. Our algorithm learns a joint model on the (same) labeled dataset used for LF induction along with any unlabeled data in a semi-supervised manner.
  arXiv  Detail & Related papers  (2021-09-23T14:42:46Z)
• Self-Tuning for Data-Efficient Deep Learning [75.34320911480008]
  Self-Tuning is a novel approach to enable data-efficient deep learning. It unifies the exploration of labeled and unlabeled data and the transfer of a pre-trained model. It outperforms its SSL and TL counterparts on five tasks by sharp margins.
  arXiv  Detail & Related papers  (2021-02-25T14:56:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.