Single-Stage Open-world Instance Segmentation with Cross-task Consistency Regularization

• URL: http://arxiv.org/abs/2208.09023v1
• Date: Thu, 18 Aug 2022 18:55:09 GMT
• Title: Single-Stage Open-world Instance Segmentation with Cross-task Consistency Regularization
• Authors: Xizhe Xue and Dongdong Yu and Lingqiao Liu and Yu Liu and Ying Li and Zehuan Yuan and Ping Song and Mike Zheng Shou
• Abstract summary: Open-world instance segmentation aims to segment class-agnostic instances from images. This paper proposes a single-stage framework to produce a mask for each instance directly. We show that the proposed method can achieve impressive results in both fully-supervised and semi-supervised settings.
• Score: 33.434628514542375
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Open-world instance segmentation (OWIS) aims to segment class-agnostic instances from images, which has a wide range of real-world applications such as autonomous driving. Most existing approaches follow a two-stage pipeline: performing class-agnostic detection first and then class-specific mask segmentation. In contrast, this paper proposes a single-stage framework to produce a mask for each instance directly. Also, instance mask annotations could be noisy in the existing datasets; to overcome this issue, we introduce a new regularization loss. Specifically, we first train an extra branch to perform an auxiliary task of predicting foreground regions (i.e. regions belonging to any object instance), and then encourage the prediction from the auxiliary branch to be consistent with the predictions of the instance masks. The key insight is that such a cross-task consistency loss could act as an error-correcting mechanism to combat the errors in annotations. Further, we discover that the proposed cross-task consistency loss can be applied to images without any annotation, lending itself to a semi-supervised learning method. Through extensive experiments, we demonstrate that the proposed method can achieve impressive results in both fully-supervised and semi-supervised settings. Compared to SOTA methods, the proposed method significantly improves the $AP_{100}$ score by 4.75\% in UVO$\rightarrow$UVO setting and 4.05\% in COCO$\rightarrow$UVO setting. In the case of semi-supervised learning, our model learned with only 30\% labeled data, even outperforms its fully-supervised counterpart with 50\% labeled data. The code will be released soon.

Related papers

• Task-customized Masked AutoEncoder via Mixture of Cluster-conditional Experts [104.9871176044644]
  Masked Autoencoder(MAE) is a prevailing self-supervised learning method that achieves promising results in model pre-training. We propose a novel MAE-based pre-training paradigm, Mixture of Cluster-conditional Experts (MoCE) MoCE trains each expert only with semantically relevant images by using cluster-conditional gates.
  arXiv  Detail & Related papers  (2024-02-08T03:46:32Z)
• One-bit Supervision for Image Classification: Problem, Solution, and Beyond [114.95815360508395]
  This paper presents one-bit supervision, a novel setting of learning with fewer labels, for image classification. We propose a multi-stage training paradigm and incorporate negative label suppression into an off-the-shelf semi-supervised learning algorithm. In multiple benchmarks, the learning efficiency of the proposed approach surpasses that using full-bit, semi-supervised supervision.
  arXiv  Detail & Related papers  (2023-11-26T07:39:00Z)
• Rethinking Multiple Instance Learning for Whole Slide Image Classification: A Good Instance Classifier is All You Need [18.832471712088353]
  We propose an instance-level weakly supervised contrastive learning algorithm for the first time under the MIL setting. We also propose an accurate pseudo label generation method through prototype learning.
  arXiv  Detail & Related papers  (2023-07-05T12:44:52Z)
• Multi-Level Contrastive Learning for Dense Prediction Task [59.591755258395594]
  We present Multi-Level Contrastive Learning for Dense Prediction Task (MCL), an efficient self-supervised method for learning region-level feature representation for dense prediction tasks. Our method is motivated by the three key factors in detection: localization, scale consistency and recognition. Our method consistently outperforms the recent state-of-the-art methods on various datasets with significant margins.
  arXiv  Detail & Related papers  (2023-04-04T17:59:04Z)
• MaskDiff: Modeling Mask Distribution with Diffusion Probabilistic Model for Few-Shot Instance Segmentation [31.648523213206595]
  Few-shot instance segmentation extends the few-shot learning paradigm to the instance segmentation task. Conventional approaches have attempted to address the task via prototype learning, known as point estimation. We propose a novel approach, dubbed MaskDiff, which models the underlying conditional distribution of a binary mask.
  arXiv  Detail & Related papers  (2023-03-09T08:24:02Z)
• Collaborative Propagation on Multiple Instance Graphs for 3D Instance Segmentation with Single-point Supervision [63.429704654271475]
  We propose a novel weakly supervised method RWSeg that only requires labeling one object with one point. With these sparse weak labels, we introduce a unified framework with two branches to propagate semantic and instance information. Specifically, we propose a Cross-graph Competing Random Walks (CRW) algorithm that encourages competition among different instance graphs.
  arXiv  Detail & Related papers  (2022-08-10T02:14:39Z)
• FreeSOLO: Learning to Segment Objects without Annotations [191.82134817449528]
  We present FreeSOLO, a self-supervised instance segmentation framework built on top of the simple instance segmentation method SOLO. Our method also presents a novel localization-aware pre-training framework, where objects can be discovered from complicated scenes in an unsupervised manner.
  arXiv  Detail & Related papers  (2022-02-24T16:31:44Z)
• Structure-Consistent Weakly Supervised Salient Object Detection with Local Saliency Coherence [14.79639149658596]
  We propose a one-round end-to-end training approach for weakly supervised salient object detection via scribble annotations. Our method achieves a new state-of-the-art performance on six benchmarks.
  arXiv  Detail & Related papers  (2020-12-08T12:49:40Z)
• Deep Semi-supervised Knowledge Distillation for Overlapping Cervical Cell Instance Segmentation [54.49894381464853]
  We propose to leverage both labeled and unlabeled data for instance segmentation with improved accuracy by knowledge distillation. We propose a novel Mask-guided Mean Teacher framework with Perturbation-sensitive Sample Mining. Experiments show that the proposed method improves the performance significantly compared with the supervised method learned from labeled data only.
  arXiv  Detail & Related papers  (2020-07-21T13:27:09Z)
• EOLO: Embedded Object Segmentation only Look Once [0.0]
  We introduce an anchor-free and single-shot instance segmentation method, which is conceptually simple with 3 independent branches, fully convolutional and can be used by easily embedding it into mobile and embedded devices. Our method, refer as EOLO, reformulates the instance segmentation problem as predicting semantic segmentation and distinguishing overlapping objects problem, through instance center classification and 4D distance regression on each pixel. Without any bells and whistles, EOLO achieves 27.7$%$ in mask mAP under IoU50 and reaches 30 FPS on 1080Ti GPU, with a single-model and single-scale training/testing on
  arXiv  Detail & Related papers  (2020-03-31T21:22:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.