PanSR: An Object-Centric Mask Transformer for Panoptic Segmentation

• URL: http://arxiv.org/abs/2412.10589v1
• Date: Fri, 13 Dec 2024 22:12:37 GMT
• Title: PanSR: An Object-Centric Mask Transformer for Panoptic Segmentation
• Authors: Lojze Žust, Matej Kristan,
• Abstract summary: Panoptic segmentation is a fundamental task in computer vision and a crucial component for perception in autonomous vehicles. Recent mask-transformer-based methods achieve impressive performance on standard benchmarks but face significant challenges with small objects, crowded scenes and scenes exhibiting a wide range of object scales. We propose a novel method for panoptic segmentation PanSR. PanSR effectively mitigates instance merging, enhances small-object detection and increases performance in crowded scenes, delivering a notable +3.4 PQ improvement over state-of-the-art on the challenging LaRS benchmark, while reaching state-of-the-art performance on Cityscapes.
• Score: 9.713215680147583
• License:
• Abstract: Panoptic segmentation is a fundamental task in computer vision and a crucial component for perception in autonomous vehicles. Recent mask-transformer-based methods achieve impressive performance on standard benchmarks but face significant challenges with small objects, crowded scenes and scenes exhibiting a wide range of object scales. We identify several fundamental shortcomings of the current approaches: (i) the query proposal generation process is biased towards larger objects, resulting in missed smaller objects, (ii) initially well-localized queries may drift to other objects, resulting in missed detections, (iii) spatially well-separated instances may be merged into a single mask causing inconsistent and false scene interpretations. To address these issues, we rethink the individual components of the network and its supervision, and propose a novel method for panoptic segmentation PanSR. PanSR effectively mitigates instance merging, enhances small-object detection and increases performance in crowded scenes, delivering a notable +3.4 PQ improvement over state-of-the-art on the challenging LaRS benchmark, while reaching state-of-the-art performance on Cityscapes. The code and models will be publicly available at https://github.com/lojzezust/PanSR.

Related papers

• PanoSSC: Exploring Monocular Panoptic 3D Scene Reconstruction for Autonomous Driving [15.441175735210791]
  Vision-centric occupancy networks represent the surrounding environment with uniform voxels with semantics. Modern occupancy networks mainly focus on reconstructing visible voxels from object surfaces with voxel-wise semantic prediction.
  arXiv  Detail & Related papers  (2024-06-11T07:51:26Z)
• Appearance-Based Refinement for Object-Centric Motion Segmentation [85.2426540999329]
  We introduce an appearance-based refinement method that leverages temporal consistency in video streams to correct inaccurate flow-based proposals. Our approach involves a sequence-level selection mechanism that identifies accurate flow-predicted masks as exemplars. Its performance is evaluated on multiple video segmentation benchmarks, including DAVIS, YouTube, SegTrackv2, and FBMS-59.
  arXiv  Detail & Related papers  (2023-12-18T18:59:51Z)
• Small Object Detection via Coarse-to-fine Proposal Generation and Imitation Learning [52.06176253457522]
  We propose a two-stage framework tailored for small object detection based on the Coarse-to-fine pipeline and Feature Imitation learning. CFINet achieves state-of-the-art performance on the large-scale small object detection benchmarks, SODA-D and SODA-A.
  arXiv  Detail & Related papers  (2023-08-18T13:13:09Z)
• Transformation-Invariant Network for Few-Shot Object Detection in Remote Sensing Images [15.251042369061024]
  Few-shot object detection (FSOD) relies on a large amount of labeled data for training. Scale and orientation variations of objects in remote sensing images pose significant challenges to existing FSOD methods. We propose integrating a feature pyramid network and utilizing prototype features to enhance query features.
  arXiv  Detail & Related papers  (2023-03-13T02:21:38Z)
• To Make Yourself Invisible with Adversarial Semantic Contours [47.755808439588094]
  Adversarial Semantic Contour (ASC) is an estimate of a Bayesian formulation of sparse attack with a deceived prior of object contour. We show that ASC can corrupt the prediction of 9 modern detectors with different architectures. We conclude with cautions about contour being the common weakness of object detectors with various architecture.
  arXiv  Detail & Related papers  (2023-03-01T07:22:39Z)
• Self-Supervised Video Object Segmentation via Cutout Prediction and Tagging [117.73967303377381]
  We propose a novel self-supervised Video Object (VOS) approach that strives to achieve better object-background discriminability. Our approach is based on a discriminative learning loss formulation that takes into account both object and background information. Our proposed approach, CT-VOS, achieves state-of-the-art results on two challenging benchmarks: DAVIS-2017 and Youtube-VOS.
  arXiv  Detail & Related papers  (2022-04-22T17:53:27Z)
• AF$_2$: Adaptive Focus Framework for Aerial Imagery Segmentation [86.44683367028914]
  Aerial imagery segmentation has some unique challenges, the most critical one among which lies in foreground-background imbalance. We propose Adaptive Focus Framework (AF$), which adopts a hierarchical segmentation procedure and focuses on adaptively utilizing multi-scale representations. AF$ has significantly improved the accuracy on three widely used aerial benchmarks, as fast as the mainstream method.
  arXiv  Detail & Related papers  (2022-02-18T10:14:45Z)
• Semantic Attention and Scale Complementary Network for Instance Segmentation in Remote Sensing Images [54.08240004593062]
  We propose an end-to-end multi-category instance segmentation model, which consists of a Semantic Attention (SEA) module and a Scale Complementary Mask Branch (SCMB) SEA module contains a simple fully convolutional semantic segmentation branch with extra supervision to strengthen the activation of interest instances on the feature map. SCMB extends the original single mask branch to trident mask branches and introduces complementary mask supervision at different scales.
  arXiv  Detail & Related papers  (2021-07-25T08:53:59Z)
• Superpixel-based Refinement for Object Proposal Generation [3.1981440103815717]
  We introduce a new superpixel-based refinement approach on top of the state-of-the-art object proposal system AttentionMask. Our experiments show an improvement of up to 26.4% in terms of average recall compared to original AttentionMask.
  arXiv  Detail & Related papers  (2021-01-12T16:06:48Z)
• Self-supervised Human Detection and Segmentation via Multi-view Consensus [116.92405645348185]
  We propose a multi-camera framework in which geometric constraints are embedded in the form of multi-view consistency during training. We show that our approach outperforms state-of-the-art self-supervised person detection and segmentation techniques on images that visually depart from those of standard benchmarks.
  arXiv  Detail & Related papers  (2020-12-09T15:47:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.