Stereo Superpixel Segmentation Via Decoupled Dynamic Spatial-Embedding Fusion Network

• URL: http://arxiv.org/abs/2208.08145v1
• Date: Wed, 17 Aug 2022 08:22:50 GMT
• Title: Stereo Superpixel Segmentation Via Decoupled Dynamic Spatial-Embedding Fusion Network
• Authors: Hua Li and Junyan Liang and Ruiqi Wu and Runmin Cong and Junhui Wu and Sam Tak Wu Kwong
• Abstract summary: We propose a stereo superpixel segmentation method with a decoupling mechanism of spatial information in this work. To decouple stereo disparity information and spatial information, the spatial information is temporarily removed before fusing the features of stereo image pairs. Our method can achieve the state-of-the-art performance on the KITTI2015 and Cityscapes datasets, and also verify the efficiency when applied in salient object detection on NJU2K dataset.
• Score: 17.05076034398913
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Stereo superpixel segmentation aims at grouping the discretizing pixels into perceptual regions through left and right views more collaboratively and efficiently. Existing superpixel segmentation algorithms mostly utilize color and spatial features as input, which may impose strong constraints on spatial information while utilizing the disparity information in terms of stereo image pairs. To alleviate this issue, we propose a stereo superpixel segmentation method with a decoupling mechanism of spatial information in this work. To decouple stereo disparity information and spatial information, the spatial information is temporarily removed before fusing the features of stereo image pairs, and a decoupled stereo fusion module (DSFM) is proposed to handle the stereo features alignment as well as occlusion problems. Moreover, since the spatial information is vital to superpixel segmentation, we further design a dynamic spatiality embedding module (DSEM) to re-add spatial information, and the weights of spatial information will be adaptively adjusted through the dynamic fusion (DF) mechanism in DSEM for achieving a finer segmentation. Comprehensive experimental results demonstrate that our method can achieve the state-of-the-art performance on the KITTI2015 and Cityscapes datasets, and also verify the efficiency when applied in salient object detection on NJU2K dataset. The source code will be available publicly after paper is accepted.

Related papers

• Remote Sensing Image Segmentation Using Vision Mamba and Multi-Scale Multi-Frequency Feature Fusion [9.098711843118629]
  This paper introduces state space model (SSM) and proposes a novel hybrid semantic segmentation network based on vision Mamba (CVMH-UNet) This method designs a cross-scanning visual state space block (CVSSBlock) that uses cross 2D scanning (CS2D) to fully capture global information from multiple directions. By incorporating convolutional neural network branches to overcome the constraints of Vision Mamba (VMamba) in acquiring local information, this approach facilitates a comprehensive analysis of both global and local features.
  arXiv  Detail & Related papers  (2024-10-08T02:17:38Z)
• Learning Accurate and Enriched Features for Stereo Image Super-Resolution [0.0]
  Stereo image super-resolution (stereoSR) aims to enhance the quality of super-resolution results by incorporating complementary information from an alternative view. We propose a mixed-scale selective fusion network (MSSFNet) to preserve precise spatial details and incorporate abundant contextual information. MSSFNet achieves significant improvements over state-of-the-art approaches on both quantitative and qualitative evaluations.
  arXiv  Detail & Related papers  (2024-06-23T03:34:17Z)
• Low-light Stereo Image Enhancement and De-noising in the Low-frequency Information Enhanced Image Space [5.1569866461097185]
  Methods are proposed to perform enhancement and de-noising simultaneously. Low-frequency information enhanced module (IEM) is proposed to suppress noise and produce a new image space. Cross-channel and spatial context information mining module (CSM) is proposed to encode long-range spatial dependencies. An encoder-decoder structure is constructed, incorporating cross-view and cross-scale feature interactions.
  arXiv  Detail & Related papers  (2024-01-15T15:03:32Z)
• SpACNN-LDVAE: Spatial Attention Convolutional Latent Dirichlet Variational Autoencoder for Hyperspectral Pixel Unmixing [1.8024397171920885]
  This work extends the Latent Dirichlet Variational Autoencoder (LDVAE) pixel unmixing scheme by taking into account local spatial context. The proposed method uses an isotropic convolutional neural network with spatial attention to encode pixels as a dirichlet distribution over endmembers.
  arXiv  Detail & Related papers  (2023-11-17T18:45:00Z)
• Object Detection in Hyperspectral Image via Unified Spectral-Spatial Feature Aggregation [55.9217962930169]
  We present S2ADet, an object detector that harnesses the rich spectral and spatial complementary information inherent in hyperspectral images. S2ADet surpasses existing state-of-the-art methods, achieving robust and reliable results.
  arXiv  Detail & Related papers  (2023-06-14T09:01:50Z)
• Alignment-free HDR Deghosting with Semantics Consistent Transformer [76.91669741684173]
  High dynamic range imaging aims to retrieve information from multiple low-dynamic range inputs to generate realistic output. Existing methods often focus on the spatial misalignment across input frames caused by the foreground and/or camera motion. We propose a novel alignment-free network with a Semantics Consistent Transformer (SCTNet) with both spatial and channel attention modules.
  arXiv  Detail & Related papers  (2023-05-29T15:03:23Z)
• SAWU-Net: Spatial Attention Weighted Unmixing Network for Hyperspectral Images [91.20864037082863]
  We propose a spatial attention weighted unmixing network, dubbed as SAWU-Net, which learns a spatial attention network and a weighted unmixing network in an end-to-end manner. In particular, we design a spatial attention module, which consists of a pixel attention block and a window attention block to efficiently model pixel-based spectral information and patch-based spatial information. Experimental results on real and synthetic datasets demonstrate the better accuracy and superiority of SAWU-Net.
  arXiv  Detail & Related papers  (2023-04-22T05:22:50Z)
• Decoupled-and-Coupled Networks: Self-Supervised Hyperspectral Image Super-Resolution with Subpixel Fusion [67.35540259040806]
  We propose a subpixel-level HS super-resolution framework by devising a novel decoupled-and-coupled network, called DCNet. As the name suggests, DC-Net first decouples the input into common (or cross-sensor) and sensor-specific components. We append a self-supervised learning module behind the CSU net by guaranteeing the material consistency to enhance the detailed appearances of the restored HS product.
  arXiv  Detail & Related papers  (2022-05-07T23:40:36Z)
• Superpixel Segmentation Based on Spatially Constrained Subspace Clustering [57.76302397774641]
  We consider each representative region with independent semantic information as a subspace, and formulate superpixel segmentation as a subspace clustering problem. We show that a simple integration of superpixel segmentation with the conventional subspace clustering does not effectively work due to the spatial correlation of the pixels. We propose a novel convex locality-constrained subspace clustering model that is able to constrain the spatial adjacent pixels with similar attributes to be clustered into a superpixel.
  arXiv  Detail & Related papers  (2020-12-11T06:18:36Z)
• Spatial Information Guided Convolution for Real-Time RGBD Semantic Segmentation [79.78416804260668]
  We propose Spatial information guided Convolution (S-Conv), which allows efficient RGB feature and 3D spatial information integration. S-Conv is competent to infer the sampling offset of the convolution kernel guided by the 3D spatial information. We further embed S-Conv into a semantic segmentation network, called Spatial information Guided convolutional Network (SGNet)
  arXiv  Detail & Related papers  (2020-04-09T13:38:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.