Ensembling Instance and Semantic Segmentation for Panoptic Segmentation

• URL: http://arxiv.org/abs/2304.10326v1
• Date: Thu, 20 Apr 2023 14:02:01 GMT
• Title: Ensembling Instance and Semantic Segmentation for Panoptic Segmentation
• Authors: Mehmet Yildirim, Yogesh Langhe
• Abstract summary: Methods first performs instance segmentation and semantic segmentation separately, then combines the two to generate panoptic segmentation results. We add several expert models of Mask R-CNN in instance segmentation to tackle the data imbalance problem in the training data. In semantic segmentation, we trained several models with various backbones and use an ensemble strategy which further boosts the segmentation results.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: We demonstrate our solution for the 2019 COCO panoptic segmentation task. Our method first performs instance segmentation and semantic segmentation separately, then combines the two to generate panoptic segmentation results. To enhance the performance, we add several expert models of Mask R-CNN in instance segmentation to tackle the data imbalance problem in the training data; also HTC model is adopted yielding our best instance segmentation results. In semantic segmentation, we trained several models with various backbones and use an ensemble strategy which further boosts the segmentation results. In the end, we analyze various combinations of instance and semantic segmentation, and report on their performance for the final panoptic segmentation results. Our best model achieves $PQ$ 47.1 on 2019 COCO panoptic test-dev data.

Related papers

• OMG-Seg: Is One Model Good Enough For All Segmentation? [83.17068644513144]
  OMG-Seg is a transformer-based encoder-decoder architecture with task-specific queries and outputs. We show that OMG-Seg can support over ten distinct segmentation tasks and yet significantly reduce computational and parameter overhead.
  arXiv  Detail & Related papers  (2024-01-18T18:59:34Z)
• Unsupervised Universal Image Segmentation [59.0383635597103]
  We propose an Unsupervised Universal model (U2Seg) adept at performing various image segmentation tasks. U2Seg generates pseudo semantic labels for these segmentation tasks via leveraging self-supervised models. We then self-train the model on these pseudo semantic labels, yielding substantial performance gains.
  arXiv  Detail & Related papers  (2023-12-28T18:59:04Z)
• Synthetic Instance Segmentation from Semantic Image Segmentation Masks [15.477053085267404]
  We propose a novel paradigm called Synthetic Instance (SISeg) SISeg instance segmentation results by leveraging image masks generated by existing semantic segmentation models. In other words, the proposed model does not need extra manpower or higher computational expenses.
  arXiv  Detail & Related papers  (2023-08-02T05:13:02Z)
• Semantic-SAM: Segment and Recognize Anything at Any Granularity [83.64686655044765]
  We introduce Semantic-SAM, a universal image segmentation model to enable segment and recognize anything at any desired granularity. We consolidate multiple datasets across three granularities and introduce decoupled classification for objects and parts. For the multi-granularity capability, we propose a multi-choice learning scheme during training, enabling each click to generate masks at multiple levels.
  arXiv  Detail & Related papers  (2023-07-10T17:59:40Z)
• Instance Segmentation of Unlabeled Modalities via Cyclic Segmentation GAN [27.936725483892076]
  We propose a novel Cyclic Generative Adrial Network (CySGAN) that conducts image translation and instance segmentation jointly. We benchmark our approach on the task of 3D neuronal nuclei segmentation with annotated electron microscopy (EM) images and unlabeled expansion microscopy (ExM) data.
  arXiv  Detail & Related papers  (2022-04-06T20:46:39Z)
• Simultaneous Semantic and Instance Segmentation for Colon Nuclei Identification and Counting [3.582485486382699]
  We present a solution framed as a simultaneous semantic and instance segmentation framework. Our solution is part of the Colon Nuclei Identification and Counting (CoNIC) Challenge.
  arXiv  Detail & Related papers  (2022-03-01T00:25:06Z)
• SOLO: A Simple Framework for Instance Segmentation [84.00519148562606]
  "instance categories" assigns categories to each pixel within an instance according to the instance's location. "SOLO" is a simple, direct, and fast framework for instance segmentation with strong performance. Our approach achieves state-of-the-art results for instance segmentation in terms of both speed and accuracy.
  arXiv  Detail & Related papers  (2021-06-30T09:56:54Z)
• Unifying Instance and Panoptic Segmentation with Dynamic Rank-1 Convolutions [109.2706837177222]
  DR1Mask is the first panoptic segmentation framework that exploits a shared feature map for both instance and semantic segmentation. As a byproduct, DR1Mask is 10% faster and 1 point in mAP more accurate than previous state-of-the-art instance segmentation network BlendMask.
  arXiv  Detail & Related papers  (2020-11-19T12:42:10Z)
• Learning Panoptic Segmentation from Instance Contours [9.347742071428918]
  Panopticpixel aims to provide an understanding of background (stuff) and instances of objects (things) at a pixel level. It combines the separate tasks of semantic segmentation (level classification) and instance segmentation to build a single unified scene understanding task. We present a fully convolution neural network that learns instance segmentation from semantic segmentation and instance contours.
  arXiv  Detail & Related papers  (2020-10-16T03:05:48Z)
• The Devil is in Classification: A Simple Framework for Long-tail Object Detection and Instance Segmentation [93.17367076148348]
  We investigate performance drop of the state-of-the-art two-stage instance segmentation model Mask R-CNN on the recent long-tail LVIS dataset. We unveil that a major cause is the inaccurate classification of object proposals. We propose a simple calibration framework to more effectively alleviate classification head bias with a bi-level class balanced sampling approach.
  arXiv  Detail & Related papers  (2020-07-23T12:49:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.