MetaSeg: Content-Aware Meta-Net for Omni-Supervised Semantic Segmentation

• URL: http://arxiv.org/abs/2401.11738v1
• Date: Mon, 22 Jan 2024 07:31:52 GMT
• Title: MetaSeg: Content-Aware Meta-Net for Omni-Supervised Semantic Segmentation
• Authors: Shenwang Jiang, Jianan Li, Ying Wang, Wenxuan Wu, Jizhou Zhang, Bo Huang, Tingfa Xu
• Abstract summary: Noisy labels, inevitably existing in pseudo segmentation labels generated from weak object-level annotations, severely hampers model optimization for semantic segmentation. Inspired by recent advances in meta learning, we argue that rather than struggling to tolerate noise hidden behind clean labels passively, a more feasible solution would be to find out the noisy regions actively. We present a novel meta learning based semantic segmentation method, MetaSeg, that comprises a primary content-aware meta-net (CAM-Net) to sever as a noise indicator for an arbitrary segmentation model counterpart.
• Score: 17.59676962334776
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Noisy labels, inevitably existing in pseudo segmentation labels generated from weak object-level annotations, severely hampers model optimization for semantic segmentation. Previous works often rely on massive hand-crafted losses and carefully-tuned hyper-parameters to resist noise, suffering poor generalization capability and high model complexity. Inspired by recent advances in meta learning, we argue that rather than struggling to tolerate noise hidden behind clean labels passively, a more feasible solution would be to find out the noisy regions actively, so as to simply ignore them during model optimization. With this in mind, this work presents a novel meta learning based semantic segmentation method, MetaSeg, that comprises a primary content-aware meta-net (CAM-Net) to sever as a noise indicator for an arbitrary segmentation model counterpart. Specifically, CAM-Net learns to generate pixel-wise weights to suppress noisy regions with incorrect pseudo labels while highlighting clean ones by exploiting hybrid strengthened features from image content, providing straightforward and reliable guidance for optimizing the segmentation model. Moreover, to break the barrier of time-consuming training when applying meta learning to common large segmentation models, we further present a new decoupled training strategy that optimizes different model layers in a divide-and-conquer manner. Extensive experiments on object, medical, remote sensing and human segmentation shows that our method achieves superior performance, approaching that of fully supervised settings, which paves a new promising way for omni-supervised semantic segmentation.

Related papers

• Precision matters: Precision-aware ensemble for weakly supervised semantic segmentation [14.931551206723041]
  Weakly Supervised Semantic (WSSS) employs weak supervision, such as image-level labels, to train the segmentation model. We propose ORANDNet, an advanced ensemble approach tailored for WSSS.
  arXiv  Detail & Related papers  (2024-06-28T03:58:02Z)
• SmooSeg: Smoothness Prior for Unsupervised Semantic Segmentation [27.367986520072147]
  Unsupervised semantic segmentation is a challenging task that segments images into semantic groups without manual annotation. We propose a novel approach called SmooSeg that harnesses self-supervised learning methods to model the closeness relationships among observations as smoothness signals. Our SmooSeg significantly outperforms STEGO in terms of pixel accuracy on three datasets.
  arXiv  Detail & Related papers  (2023-10-27T03:29:25Z)
• MixReorg: Cross-Modal Mixed Patch Reorganization is a Good Mask Learner for Open-World Semantic Segmentation [110.09800389100599]
  We propose MixReorg, a novel and straightforward pre-training paradigm for semantic segmentation. Our approach involves generating fine-grained patch-text pairs data by mixing image patches while preserving the correspondence between patches and text. With MixReorg as a mask learner, conventional text-supervised semantic segmentation models can achieve highly generalizable pixel-semantic alignment ability.
  arXiv  Detail & Related papers  (2023-08-09T09:35:16Z)
• Towards Reliable Evaluation and Fast Training of Robust Semantic Segmentation Models [47.03411822627386]
  We propose several problem-specific novel attacks minimizing different metrics in accuracy and mIoU. Surprisingly, existing attempts of adversarial training for semantic segmentation models turn out to be weak or even completely non-robust. We show how recently proposed robust ImageNet backbones can be used to obtain adversarially robust semantic segmentation models with up to six times less training time for PASCAL-VOC and the more challenging ADE20k.
  arXiv  Detail & Related papers  (2023-06-22T14:56:06Z)
• Multi-Granularity Denoising and Bidirectional Alignment for Weakly Supervised Semantic Segmentation [75.32213865436442]
  We propose an end-to-end multi-granularity denoising and bidirectional alignment (MDBA) model to alleviate the noisy label and multi-class generalization issues. The MDBA model can reach the mIoU of 69.5% and 70.2% on validation and test sets for the PASCAL VOC 2012 dataset.
  arXiv  Detail & Related papers  (2023-05-09T03:33:43Z)
• Semantics-Aware Dynamic Localization and Refinement for Referring Image Segmentation [102.25240608024063]
  Referring image segments an image from a language expression. We develop an algorithm that shifts from being localization-centric to segmentation-language. Compared to its counterparts, our method is more versatile yet effective.
  arXiv  Detail & Related papers  (2023-03-11T08:42:40Z)
• Incremental Learning in Semantic Segmentation from Image Labels [18.404068463921426]
  Existing semantic segmentation approaches achieve impressive results, but struggle to update their models incrementally as new categories are uncovered. This paper proposes a novel framework for Weakly Incremental Learning for Semantics, that aims at learning to segment new classes from cheap and largely available image-level labels. As opposed to existing approaches, that need to generate pseudo-labels offline, we use an auxiliary classifier, trained with image-level labels and regularized by the segmentation model, to obtain pseudo-supervision online and update the model incrementally.
  arXiv  Detail & Related papers  (2021-12-03T12:47:12Z)
• Half-Real Half-Fake Distillation for Class-Incremental Semantic Segmentation [84.1985497426083]
  convolutional neural networks are ill-equipped for incremental learning. New classes are available but the initial training data is not retained. We try to address this issue by "inverting" the trained segmentation network to synthesize input images starting from random noise.
  arXiv  Detail & Related papers  (2021-04-02T03:47:16Z)
• Group-Wise Semantic Mining for Weakly Supervised Semantic Segmentation [49.90178055521207]
  This work addresses weakly supervised semantic segmentation (WSSS), with the goal of bridging the gap between image-level annotations and pixel-level segmentation. We formulate WSSS as a novel group-wise learning task that explicitly models semantic dependencies in a group of images to estimate more reliable pseudo ground-truths. In particular, we devise a graph neural network (GNN) for group-wise semantic mining, wherein input images are represented as graph nodes.
  arXiv  Detail & Related papers  (2020-12-09T12:40:13Z)
• Improving Semantic Segmentation via Decoupled Body and Edge Supervision [89.57847958016981]
  Existing semantic segmentation approaches either aim to improve the object's inner consistency by modeling the global context, or refine objects detail along their boundaries by multi-scale feature fusion. In this paper, a new paradigm for semantic segmentation is proposed. Our insight is that appealing performance of semantic segmentation requires textitexplicitly modeling the object textitbody and textitedge, which correspond to the high and low frequency of the image. We show that the proposed framework with various baselines or backbone networks leads to better object inner consistency and object boundaries.
  arXiv  Detail & Related papers  (2020-07-20T12:11:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.