Full or Weak annotations? An adaptive strategy for budget-constrained annotation campaigns

• URL: http://arxiv.org/abs/2303.11678v1
• Date: Tue, 21 Mar 2023 08:41:54 GMT
• Title: Full or Weak annotations? An adaptive strategy for budget-constrained annotation campaigns
• Authors: Javier Gamazo Tejero, Martin S. Zinkernagel, Sebastian Wolf, Raphael Sznitman and Pablo M\'arquez Neila
• Abstract summary: We propose a novel approach to determine annotation strategies for segmentation datasets. Our method sequentially determines proportions of segmentation and classification annotations to collect for budget-fractions. We show in our experiments that our approach yields annotations that perform very close to the optimal for a number of different annotation budgets and datasets.
• Score: 3.1318537187387787
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Annotating new datasets for machine learning tasks is tedious, time-consuming, and costly. For segmentation applications, the burden is particularly high as manual delineations of relevant image content are often extremely expensive or can only be done by experts with domain-specific knowledge. Thanks to developments in transfer learning and training with weak supervision, segmentation models can now also greatly benefit from annotations of different kinds. However, for any new domain application looking to use weak supervision, the dataset builder still needs to define a strategy to distribute full segmentation and other weak annotations. Doing so is challenging, however, as it is a priori unknown how to distribute an annotation budget for a given new dataset. To this end, we propose a novel approach to determine annotation strategies for segmentation datasets, whereby estimating what proportion of segmentation and classification annotations should be collected given a fixed budget. To do so, our method sequentially determines proportions of segmentation and classification annotations to collect for budget-fractions by modeling the expected improvement of the final segmentation model. We show in our experiments that our approach yields annotations that perform very close to the optimal for a number of different annotation budgets and datasets.

Related papers

• Selective Annotation via Data Allocation: These Data Should Be Triaged to Experts for Annotation Rather Than the Model [42.70608373297776]
  We propose a selective annotation framework called SANT. It effectively takes advantage of both the triage-to-human and triage-to-model data through the proposed error-aware triage and bi-weighting mechanisms. Experimental results show that SANT consistently outperforms other baselines, leading to higher-quality annotation through its proper allocation of data to both expert and model workers.
  arXiv  Detail & Related papers  (2024-05-20T14:52:05Z)
• Annotation-Efficient Polyp Segmentation via Active Learning [45.59503015577479]
  We propose a deep active learning framework for annotation-efficient polyp segmentation. In practice, we measure the uncertainty of each sample by examining the similarity between features masked by the prediction map of the polyp and the background area. We show that our proposed method achieved state-of-the-art performance compared to other competitors on both a public dataset and a large-scale in-house dataset.
  arXiv  Detail & Related papers  (2024-03-21T12:25:17Z)
• Data-efficient Active Learning for Structured Prediction with Partial Annotation and Self-Training [16.740101757982828]
  We propose a pragmatic method that reduces the annotation cost for structured label spaces using active learning. Our approach leverages partial annotation, which reduces labeling costs by selecting only the most informative sub-structures for annotation. We also utilize self-training to incorporate the current model's automatic predictions as pseudo-labels for un-annotated sub-structures.
  arXiv  Detail & Related papers  (2023-05-22T01:58:42Z)
• Knowledge Combination to Learn Rotated Detection Without Rotated Annotation [53.439096583978504]
  Rotated bounding boxes drastically reduce output ambiguity of elongated objects. Despite the effectiveness, rotated detectors are not widely employed. We propose a framework that allows the model to predict precise rotated boxes.
  arXiv  Detail & Related papers  (2023-04-05T03:07:36Z)
• Urban Scene Semantic Segmentation with Low-Cost Coarse Annotation [107.72926721837726]
  coarse annotation is a low-cost but highly effective alternative for training semantic segmentation models. We propose a coarse-to-fine self-training framework that generates pseudo labels for unlabeled regions of coarsely annotated data. Our method achieves a significantly better performance vs annotation cost tradeoff, yielding a comparable performance to fully annotated data with only a small fraction of the annotation budget.
  arXiv  Detail & Related papers  (2022-12-15T15:43:42Z)
• Active Pointly-Supervised Instance Segmentation [106.38955769817747]
  We present an economic active learning setting, named active pointly-supervised instance segmentation (APIS) APIS starts with box-level annotations and iteratively samples a point within the box and asks if it falls on the object. The model developed with these strategies yields consistent performance gain on the challenging MS-COCO dataset.
  arXiv  Detail & Related papers  (2022-07-23T11:25:24Z)
• Learning Debiased and Disentangled Representations for Semantic Segmentation [52.35766945827972]
  We propose a model-agnostic and training scheme for semantic segmentation. By randomly eliminating certain class information in each training iteration, we effectively reduce feature dependencies among classes. Models trained with our approach demonstrate strong results on multiple semantic segmentation benchmarks.
  arXiv  Detail & Related papers  (2021-10-31T16:15:09Z)
• Field-Guide-Inspired Zero-Shot Learning [39.10156282444968]
  We introduce a new field-guide-inspired approach to zero-shot annotation where the learner model interactively asks for the most useful attributes that define a class. We evaluate our method on classification benchmarks with attribute annotations like CUB, SUN, and AWA2 and show that our model achieves the performance of a model with full annotations at the cost of a significantly fewer number of annotations.
  arXiv  Detail & Related papers  (2021-08-24T21:36:05Z)
• Flip Learning: Erase to Segment [65.84901344260277]
  Weakly-supervised segmentation (WSS) can help reduce time-consuming and cumbersome manual annotation. We propose a novel and general WSS framework called Flip Learning, which only needs the box annotation. Our proposed approach achieves competitive performance and shows great potential to narrow the gap between fully-supervised and weakly-supervised learning.
  arXiv  Detail & Related papers  (2021-08-02T09:56:10Z)
• Deep Semi-supervised Knowledge Distillation for Overlapping Cervical Cell Instance Segmentation [54.49894381464853]
  We propose to leverage both labeled and unlabeled data for instance segmentation with improved accuracy by knowledge distillation. We propose a novel Mask-guided Mean Teacher framework with Perturbation-sensitive Sample Mining. Experiments show that the proposed method improves the performance significantly compared with the supervised method learned from labeled data only.
  arXiv  Detail & Related papers  (2020-07-21T13:27:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.