BANet: Bidirectional Aggregation Network with Occlusion Handling for Panoptic Segmentation

• URL: http://arxiv.org/abs/2003.14031v1
• Date: Tue, 31 Mar 2020 08:57:14 GMT
• Title: BANet: Bidirectional Aggregation Network with Occlusion Handling for Panoptic Segmentation
• Authors: Yifeng Chen, Guangchen Lin, Songyuan Li, Bourahla Omar, Yiming Wu, Fangfang Wang, Junyi Feng, Mingliang Xu, and Xi Li
• Abstract summary: Panoptic segmentation aims to perform instance segmentation for foreground instances and semantic segmentation for background stuff simultaneously. We propose a novel deep panoptic segmentation scheme based on a bidirectional learning pipeline. The experimental results on COCO panoptic benchmark validate the effectiveness of our proposed method.
• Score: 30.008473359758632
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Panoptic segmentation aims to perform instance segmentation for foreground instances and semantic segmentation for background stuff simultaneously. The typical top-down pipeline concentrates on two key issues: 1) how to effectively model the intrinsic interaction between semantic segmentation and instance segmentation, and 2) how to properly handle occlusion for panoptic segmentation. Intuitively, the complementarity between semantic segmentation and instance segmentation can be leveraged to improve the performance. Besides, we notice that using detection/mask scores is insufficient for resolving the occlusion problem. Motivated by these observations, we propose a novel deep panoptic segmentation scheme based on a bidirectional learning pipeline. Moreover, we introduce a plug-and-play occlusion handling algorithm to deal with the occlusion between different object instances. The experimental results on COCO panoptic benchmark validate the effectiveness of our proposed method. Codes will be released soon at https://github.com/Mooonside/BANet.

Related papers

• Unsupervised Universal Image Segmentation [59.0383635597103]
  We propose an Unsupervised Universal model (U2Seg) adept at performing various image segmentation tasks. U2Seg generates pseudo semantic labels for these segmentation tasks via leveraging self-supervised models. We then self-train the model on these pseudo semantic labels, yielding substantial performance gains.
  arXiv  Detail & Related papers  (2023-12-28T18:59:04Z)
• SegRefiner: Towards Model-Agnostic Segmentation Refinement with Discrete Diffusion Process [102.18226145874007]
  We propose a model-agnostic solution called SegRefiner to enhance the quality of object masks produced by different segmentation models. SegRefiner takes coarse masks as inputs and refines them using a discrete diffusion process. It consistently improves both the segmentation metrics and boundary metrics across different types of coarse masks.
  arXiv  Detail & Related papers  (2023-12-19T18:53:47Z)
• Label-efficient Segmentation via Affinity Propagation [27.016747627689288]
  Weakly-supervised segmentation with label-efficient sparse annotations has attracted increasing research attention to reduce the cost of laborious pixel-wise labeling process. We formulate the affinity modeling as an affinity propagation process, and propose a local and a global pairwise affinity terms to generate accurate soft pseudo labels. An efficient algorithm is also developed to reduce significantly the computational cost.
  arXiv  Detail & Related papers  (2023-10-16T15:54:09Z)
• A Simple Framework for Open-Vocabulary Segmentation and Detection [85.21641508535679]
  We present OpenSeeD, a simple Open-vocabulary and Detection framework that jointly learns from different segmentation and detection datasets. We first introduce a pre-trained text encoder to encode all the visual concepts in two tasks and learn a common semantic space for them. After pre-training, our model exhibits competitive or stronger zero-shot transferability for both segmentation and detection.
  arXiv  Detail & Related papers  (2023-03-14T17:58:34Z)
• Proposal-free Lidar Panoptic Segmentation with Pillar-level Affinity [10.378476897786571]
  We jointly optimize semantic segmentation and class-agnostic instance classification in a single network. Our experiments on nuScenes dataset show that our approach outperforms previous proposal-free methods.
  arXiv  Detail & Related papers  (2022-04-19T08:39:37Z)
• SOLO: A Simple Framework for Instance Segmentation [84.00519148562606]
  "instance categories" assigns categories to each pixel within an instance according to the instance's location. "SOLO" is a simple, direct, and fast framework for instance segmentation with strong performance. Our approach achieves state-of-the-art results for instance segmentation in terms of both speed and accuracy.
  arXiv  Detail & Related papers  (2021-06-30T09:56:54Z)
• Deep Occlusion-Aware Instance Segmentation with Overlapping BiLayers [72.38919601150175]
  We propose Bilayer Convolutional Network (BCNet) to segment highly-overlapping objects. BCNet detects the occluding objects (occluder) and the bottom GCN layer infers partially occluded instance (occludee)
  arXiv  Detail & Related papers  (2021-03-23T06:25:42Z)
• PointFlow: Flowing Semantics Through Points for Aerial Image Segmentation [96.76882806139251]
  We propose a point-wise affinity propagation module based on the Feature Pyramid Network (FPN) framework, named PointFlow. Rather than dense affinity learning, a sparse affinity map is generated upon selected points between the adjacent features. Experimental results on three different aerial segmentation datasets suggest that the proposed method is more effective and efficient than state-of-the-art general semantic segmentation methods.
  arXiv  Detail & Related papers  (2021-03-11T09:42:32Z)
• Learning Panoptic Segmentation from Instance Contours [9.347742071428918]
  Panopticpixel aims to provide an understanding of background (stuff) and instances of objects (things) at a pixel level. It combines the separate tasks of semantic segmentation (level classification) and instance segmentation to build a single unified scene understanding task. We present a fully convolution neural network that learns instance segmentation from semantic segmentation and instance contours.
  arXiv  Detail & Related papers  (2020-10-16T03:05:48Z)
• Towards Bounding-Box Free Panoptic Segmentation [16.4548904544277]
  We introduce a new Bounding-Box Free Network (BBFNet) for panoptic segmentation. BBFNet predicts coarse watershed levels and uses them to detect large instance candidates where boundaries are well defined. For smaller instances, whose boundaries are less reliable, BBFNet also predicts instance centers by means of Hough voting followed by mean-shift to reliably detect small objects.
  arXiv  Detail & Related papers  (2020-02-18T16:34:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.