The Accuracy Cost of Weakness: A Theoretical Analysis of Fixed-Segment Weak Labeling for Events in Time

• URL: http://arxiv.org/abs/2502.09363v1
• Date: Thu, 13 Feb 2025 14:31:49 GMT
• Title: The Accuracy Cost of Weakness: A Theoretical Analysis of Fixed-Segment Weak Labeling for Events in Time
• Authors: John Martinsson, Olof Mogren, Tuomas Virtanen, Maria Sandsten,
• Abstract summary: We model a common weak labeling process where annotators assign presence or absence labels to fixed-length data segments. We compare this fixed-length labeling approach with an oracle method that uses the true event activations to construct the segments. Our findings provide a theoretical justification for adaptive weak labeling strategies that mimic the oracle process.
• Score: 11.312115846980602
• License:
• Abstract: Accurate labels are critical for deriving robust machine learning models. Labels are used to train supervised learning models and to evaluate most machine learning paradigms. In this paper, we model the accuracy and cost of a common weak labeling process where annotators assign presence or absence labels to fixed-length data segments for a given event class. The annotator labels a segment as "present" if it sufficiently covers an event from that class, e.g., a birdsong sound event in audio data. We analyze how the segment length affects the label accuracy and the required number of annotations, and compare this fixed-length labeling approach with an oracle method that uses the true event activations to construct the segments. Furthermore, we quantify the gap between these methods and verify that in most realistic scenarios the oracle method is better than the fixed-length labeling method in both accuracy and cost. Our findings provide a theoretical justification for adaptive weak labeling strategies that mimic the oracle process, and a foundation for optimizing weak labeling processes in sequence labeling tasks.

Related papers

• Mixed Blessing: Class-Wise Embedding guided Instance-Dependent Partial Label Learning [53.64180787439527]
  In partial label learning (PLL), every sample is associated with a candidate label set comprising the ground-truth label and several noisy labels. For the first time, we create class-wise embeddings for each sample, which allow us to explore the relationship of instance-dependent noisy labels. To reduce the high label ambiguity, we introduce the concept of class prototypes containing global feature information.
  arXiv  Detail & Related papers  (2024-12-06T13:25:39Z)
• Determined Multi-Label Learning via Similarity-Based Prompt [12.428779617221366]
  In multi-label classification, each training instance is associated with multiple class labels simultaneously. To alleviate this problem, a novel labeling setting termed textitDetermined Multi-Label Learning (DMLL) is proposed.
  arXiv  Detail & Related papers  (2024-03-25T07:08:01Z)
• Drawing the Same Bounding Box Twice? Coping Noisy Annotations in Object Detection with Repeated Labels [6.872072177648135]
  We propose a novel localization algorithm that adapts well-established ground truth estimation methods. Our algorithm also shows superior performance during training on the TexBiG dataset.
  arXiv  Detail & Related papers  (2023-09-18T13:08:44Z)
• Robust Assignment of Labels for Active Learning with Sparse and Noisy Annotations [0.17188280334580192]
  Supervised classification algorithms are used to solve a growing number of real-life problems around the globe. Unfortunately, acquiring good-quality annotations for many tasks is infeasible or too expensive to be done in practice. We propose two novel annotation unification algorithms that utilize unlabeled parts of the sample space.
  arXiv  Detail & Related papers  (2023-07-25T19:40:41Z)
• Imprecise Label Learning: A Unified Framework for Learning with Various Imprecise Label Configurations [91.67511167969934]
  imprecise label learning (ILL) is a framework for the unification of learning with various imprecise label configurations. We demonstrate that ILL can seamlessly adapt to partial label learning, semi-supervised learning, noisy label learning, and, more importantly, a mixture of these settings.
  arXiv  Detail & Related papers  (2023-05-22T04:50:28Z)
• CroSel: Cross Selection of Confident Pseudo Labels for Partial-Label Learning [14.077121555766398]
  Partial-label learning (PLL) is an important weakly supervised learning problem, which allows each training example to have a candidate label set instead of a single ground-truth label. We propose a new method called CroSel, which leverages historical predictions from the model to identify true labels for most training examples.
  arXiv  Detail & Related papers  (2023-03-18T08:48:16Z)
• AutoWS: Automated Weak Supervision Framework for Text Classification [1.748907524043535]
  We propose a novel framework for increasing the efficiency of weak supervision process while decreasing the dependency on domain experts. Our method requires a small set of labeled examples per label class and automatically creates a set of labeling functions to assign noisy labels to numerous unlabeled data.
  arXiv  Detail & Related papers  (2023-02-07T07:12:05Z)
• Learning with Proper Partial Labels [87.65718705642819]
  Partial-label learning is a kind of weakly-supervised learning with inexact labels. We show that this proper partial-label learning framework includes many previous partial-label learning settings. We then derive a unified unbiased estimator of the classification risk.
  arXiv  Detail & Related papers  (2021-12-23T01:37:03Z)
• Instance-Dependent Partial Label Learning [69.49681837908511]
  Partial label learning is a typical weakly supervised learning problem. Most existing approaches assume that the incorrect labels in each training example are randomly picked as the candidate labels. In this paper, we consider instance-dependent and assume that each example is associated with a latent label distribution constituted by the real number of each label.
  arXiv  Detail & Related papers  (2021-10-25T12:50:26Z)
• Learning with Noisy Labels by Targeted Relabeling [52.0329205268734]
  Crowdsourcing platforms are often used to collect datasets for training deep neural networks. We propose an approach which reserves a fraction of annotations to explicitly relabel highly probable labeling errors.
  arXiv  Detail & Related papers  (2021-10-15T20:37:29Z)
• PseudoSeg: Designing Pseudo Labels for Semantic Segmentation [78.35515004654553]
  We present a re-design of pseudo-labeling to generate structured pseudo labels for training with unlabeled or weakly-labeled data. We demonstrate the effectiveness of the proposed pseudo-labeling strategy in both low-data and high-data regimes.
  arXiv  Detail & Related papers  (2020-10-19T17:59:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.