Benchmarking Label Noise in Instance Segmentation: Spatial Noise Matters

• URL: http://arxiv.org/abs/2406.10891v2
• Date: Tue, 18 Jun 2024 12:54:48 GMT
• Title: Benchmarking Label Noise in Instance Segmentation: Spatial Noise Matters
• Authors: Eden Grad, Moshe Kimhi, Lion Halika, Chaim Baskin,
• Abstract summary: This study sheds light on the quality of segmentation masks produced by various models. It challenges the efficacy of popular methods designed to address learning with label noise.
• Score: 2.53740603524637
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Obtaining accurate labels for instance segmentation is particularly challenging due to the complex nature of the task. Each image necessitates multiple annotations, encompassing not only the object's class but also its precise spatial boundaries. These requirements elevate the likelihood of errors and inconsistencies in both manual and automated annotation processes. By simulating different noise conditions, we provide a realistic scenario for assessing the robustness and generalization capabilities of instance segmentation models in different segmentation tasks, introducing COCO-N and Cityscapes-N. We also propose a benchmark for weakly annotation noise, dubbed COCO-WAN, which utilizes foundation models and weak annotations to simulate semi-automated annotation tools and their noisy labels. This study sheds light on the quality of segmentation masks produced by various models and challenges the efficacy of popular methods designed to address learning with label noise.

Related papers

• One-step Noisy Label Mitigation [86.57572253460125]
  Mitigating the detrimental effects of noisy labels on the training process has become increasingly critical. We propose One-step Anti-Noise (OSA), a model-agnostic noisy label mitigation paradigm. We empirically demonstrate the superiority of OSA, highlighting its enhanced training robustness, improved task transferability, ease of deployment, and reduced computational costs.
  arXiv  Detail & Related papers  (2024-10-02T18:42:56Z)
• NoisyAG-News: A Benchmark for Addressing Instance-Dependent Noise in Text Classification [7.464154519547575]
  Existing research on learning with noisy labels predominantly focuses on synthetic noise patterns. We constructed a benchmark dataset to better understand label noise in real-world text classification settings. Our findings reveal that while pre-trained models are resilient to synthetic noise, they struggle against instance-dependent noise.
  arXiv  Detail & Related papers  (2024-07-09T06:18:40Z)
• Rethinking the Value of Labels for Instance-Dependent Label Noise Learning [43.481591776038144]
  noisy labels in real-world applications often depend on both the true label and the features. In this work, we tackle instance-dependent label noise with a novel deep generative model that avoids explicitly modeling the noise transition matrix. Our algorithm leverages casual representation learning and simultaneously identifies the high-level content and style latent factors from the data.
  arXiv  Detail & Related papers  (2023-05-10T15:29:07Z)
• Learning Confident Classifiers in the Presence of Label Noise [5.829762367794509]
  This paper proposes a probabilistic model for noisy observations that allows us to build a confident classification and segmentation models. Our experiments show that our algorithm outperforms state-of-the-art solutions for the considered classification and segmentation problems.
  arXiv  Detail & Related papers  (2023-01-02T04:27:25Z)
• Learning from Noisy Labels with Coarse-to-Fine Sample Credibility Modeling [22.62790706276081]
  Training deep neural network (DNN) with noisy labels is practically challenging. Previous efforts tend to handle part or full data in a unified denoising flow. We propose a coarse-to-fine robust learning method called CREMA to handle noisy data in a divide-and-conquer manner.
  arXiv  Detail & Related papers  (2022-08-23T02:06:38Z)
• Centrality and Consistency: Two-Stage Clean Samples Identification for Learning with Instance-Dependent Noisy Labels [87.48541631675889]
  We propose a two-stage clean samples identification method. First, we employ a class-level feature clustering procedure for the early identification of clean samples. Second, for the remaining clean samples that are close to the ground truth class boundary, we propose a novel consistency-based classification method.
  arXiv  Detail & Related papers  (2022-07-29T04:54:57Z)
• SimT: Handling Open-set Noise for Domain Adaptive Semantic Segmentation [58.61946589036262]
  This paper studies a practical domain adaptive (DA) semantic segmentation problem where only pseudo-labeled target data is accessible through a black-box model. Due to the domain gap and label shift between two domains, pseudo-labeled target data contains mixed closed-set and open-set label noises. We propose a simplex noise transition matrix (SimT) to model the mixed noise distributions in DA semantic segmentation and formulate the problem as estimation of SimT.
  arXiv  Detail & Related papers  (2022-03-29T02:48:08Z)
• S3: Supervised Self-supervised Learning under Label Noise [53.02249460567745]
  In this paper we address the problem of classification in the presence of label noise. In the heart of our method is a sample selection mechanism that relies on the consistency between the annotated label of a sample and the distribution of the labels in its neighborhood in the feature space. Our method significantly surpasses previous methods on both CIFARCIFAR100 with artificial noise and real-world noisy datasets such as WebVision and ANIMAL-10N.
  arXiv  Detail & Related papers  (2021-11-22T15:49:20Z)
• A Realistic Simulation Framework for Learning with Label Noise [17.14439597393087]
  We show that this framework generates synthetic noisy labels that exhibit important characteristics of the label noise. We also benchmark several existing algorithms for learning with noisy labels. We propose a new technique, Label Quality Model (LQM), that leverages annotator features to predict and correct against noisy labels.
  arXiv  Detail & Related papers  (2021-07-23T18:53:53Z)
• Approximating Instance-Dependent Noise via Instance-Confidence Embedding [87.65718705642819]
  Label noise in multiclass classification is a major obstacle to the deployment of learning systems. We investigate the instance-dependent noise (IDN) model and propose an efficient approximation of IDN to capture the instance-specific label corruption.
  arXiv  Detail & Related papers  (2021-03-25T02:33:30Z)
• Tackling Instance-Dependent Label Noise via a Universal Probabilistic Model [80.91927573604438]
  This paper proposes a simple yet universal probabilistic model, which explicitly relates noisy labels to their instances. Experiments on datasets with both synthetic and real-world label noise verify that the proposed method yields significant improvements on robustness.
  arXiv  Detail & Related papers  (2021-01-14T05:43:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.