ScriptoriumWS: A Code Generation Assistant for Weak Supervision

• URL: http://arxiv.org/abs/2502.12366v1
• Date: Mon, 17 Feb 2025 23:07:14 GMT
• Title: ScriptoriumWS: A Code Generation Assistant for Weak Supervision
• Authors: Tzu-Heng Huang, Catherine Cao, Spencer Schoenberg, Harit Vishwakarma, Nicholas Roberts, Frederic Sala,
• Abstract summary: We argue for using code-generation models to act as coding assistants for crafting weak supervision sources. We introduce ScriptoriumWS, a weak supervision system that, when compared to hand-crafted sources, maintains accuracy and greatly improves coverage.
• Score: 16.121122576534386
• License:
• Abstract: Weak supervision is a popular framework for overcoming the labeled data bottleneck: the need to obtain labels for training data. In weak supervision, multiple noisy-but-cheap sources are used to provide guesses of the label and are aggregated to produce high-quality pseudolabels. These sources are often expressed as small programs written by domain experts -- and so are expensive to obtain. Instead, we argue for using code-generation models to act as coding assistants for crafting weak supervision sources. We study prompting strategies to maximize the quality of the generated sources, settling on a multi-tier strategy that incorporates multiple types of information. We explore how to best combine hand-written and generated sources. Using these insights, we introduce ScriptoriumWS, a weak supervision system that, when compared to hand-crafted sources, maintains accuracy and greatly improves coverage.

Related papers

• AlchemistCoder: Harmonizing and Eliciting Code Capability by Hindsight Tuning on Multi-source Data [64.69872638349922]
  We present AlchemistCoder, a series of Code LLMs with enhanced code generation and generalization capabilities fine-tuned on multi-source data. We propose incorporating the data construction process into the fine-tuning data as code comprehension tasks, including instruction evolution, data filtering, and code review.
  arXiv  Detail & Related papers  (2024-05-29T16:57:33Z)
• AutoWS: Automated Weak Supervision Framework for Text Classification [1.748907524043535]
  We propose a novel framework for increasing the efficiency of weak supervision process while decreasing the dependency on domain experts. Our method requires a small set of labeled examples per label class and automatically creates a set of labeling functions to assign noisy labels to numerous unlabeled data.
  arXiv  Detail & Related papers  (2023-02-07T07:12:05Z)
• Label Propagation with Weak Supervision [47.52032178837098]
  We introduce a novel analysis of the classical label propagation algorithm (LPA) (Zhu & Ghahramani, 2002) We provide an error bound that exploits both the local geometric properties of the underlying graph and the quality of the prior information. We demonstrate the ability of our approach on multiple benchmark weakly supervised classification tasks, showing improvements upon existing semi-supervised and weakly supervised methods.
  arXiv  Detail & Related papers  (2022-10-07T14:53:02Z)
• Generative Modeling Helps Weak Supervision (and Vice Versa) [87.62271390571837]
  We propose a model fusing weak supervision and generative adversarial networks. It captures discrete variables in the data alongside the weak supervision derived label estimate. It is the first approach to enable data augmentation through weakly supervised synthetic images and pseudolabels.
  arXiv  Detail & Related papers  (2022-03-22T20:24:21Z)
• Data Consistency for Weakly Supervised Learning [15.365232702938677]
  Training machine learning models involves using large amounts of human-annotated data. We propose a novel weak supervision algorithm that processes noisy labels, i.e., weak signals. We show that it significantly outperforms state-of-the-art weak supervision methods on both text and image classification tasks.
  arXiv  Detail & Related papers  (2022-02-08T16:48:19Z)
• Deep Transfer Learning for Multi-source Entity Linkage via Domain Adaptation [63.24594955429465]
  Multi-source entity linkage is critical in high-impact applications such as data cleaning and user stitching. AdaMEL is a deep transfer learning framework that learns generic high-level knowledge to perform multi-source entity linkage. Our framework achieves state-of-the-art results with 8.21% improvement on average over methods based on supervised learning.
  arXiv  Detail & Related papers  (2021-10-27T15:20:41Z)
• Creating Training Sets via Weak Indirect Supervision [66.77795318313372]
  Weak Supervision (WS) frameworks synthesize training labels from multiple potentially noisy supervision sources. We formulate Weak Indirect Supervision (WIS), a new research problem for automatically synthesizing training labels. We develop a probabilistic modeling approach, PLRM, which uses user-provided label relations to model and leverage indirect supervision sources.
  arXiv  Detail & Related papers  (2021-10-07T14:09:35Z)
• OpinionRank: Extracting Ground Truth Labels from Unreliable Expert Opinions with Graph-Based Spectral Ranking [2.1930130356902207]
  crowdsourcing has emerged as a popular, inexpensive, and efficient data mining solution for performing distributed label collection. We propose OpinionRank, a model-free, interpretable, graph-based spectral algorithm for integrating crowdsourced annotations into reliable labels. Our experiments show that OpinionRank performs favorably when compared against more highly parameterized algorithms.
  arXiv  Detail & Related papers  (2021-02-11T08:12:44Z)
• Bayesian Semi-supervised Crowdsourcing [71.20185379303479]
  Crowdsourcing has emerged as a powerful paradigm for efficiently labeling large datasets and performing various learning tasks. This work deals with semi-supervised crowdsourced classification, under two regimes of semi-supervision.
  arXiv  Detail & Related papers  (2020-12-20T23:18:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.