Deep Spherical Superpixels

• URL: http://arxiv.org/abs/2407.17354v1
• Date: Wed, 24 Jul 2024 15:27:21 GMT
• Title: Deep Spherical Superpixels
• Authors: Rémi Giraud, Michaël Clément,
• Abstract summary: We introduce the first deep learning-based superpixel segmentation approach tailored for omnidirectional images called DSS (for Deep Spherical Superpixels) Our methodology leverages on spherical CNN architectures and the differentiable K-means clustering paradigm for superpixels, to generate superpixels that follow the spherical geometry.
• Score: 1.104960878651584
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Over the years, the use of superpixel segmentation has become very popular in various applications, serving as a preprocessing step to reduce data size by adapting to the content of the image, regardless of its semantic content. While the superpixel segmentation of standard planar images, captured with a 90{\deg} field of view, has been extensively studied, there has been limited focus on dedicated methods to omnidirectional or spherical images, captured with a 360{\deg} field of view. In this study, we introduce the first deep learning-based superpixel segmentation approach tailored for omnidirectional images called DSS (for Deep Spherical Superpixels). Our methodology leverages on spherical CNN architectures and the differentiable K-means clustering paradigm for superpixels, to generate superpixels that follow the spherical geometry. Additionally, we propose to use data augmentation techniques specifically designed for 360{\deg} images, enabling our model to efficiently learn from a limited set of annotated omnidirectional data. Our extensive validation across two datasets demonstrates that taking into account the inherent circular geometry of such images into our framework improves the segmentation performance over traditional and deep learning-based superpixel methods. Our code is available online.

Related papers

• Superpixel Graph Contrastive Clustering with Semantic-Invariant Augmentations for Hyperspectral Images [64.72242126879503]
  Hyperspectral images (HSI) clustering is an important but challenging task. We first use 3-D and 2-D hybrid convolutional neural networks to extract the high-order spatial and spectral features of HSI. We then design a superpixel graph contrastive clustering model to learn discriminative superpixel representations.
  arXiv  Detail & Related papers  (2024-03-04T07:40:55Z)
• Superpixel Transformers for Efficient Semantic Segmentation [32.537400525407186]
  We propose a solution by leveraging the idea of superpixels, an over-segmentation of the image, and applying them with a modern transformer framework. Our method achieves state-of-the-art performance in semantic segmentation due to the rich superpixel features generated by the global self-attention mechanism.
  arXiv  Detail & Related papers  (2023-09-28T23:09:30Z)
• Probabilistic Deep Metric Learning for Hyperspectral Image Classification [91.5747859691553]
  This paper proposes a probabilistic deep metric learning framework for hyperspectral image classification. It aims to predict the category of each pixel for an image captured by hyperspectral sensors. Our framework can be readily applied to existing hyperspectral image classification methods.
  arXiv  Detail & Related papers  (2022-11-15T17:57:12Z)
• Rethinking Unsupervised Neural Superpixel Segmentation [6.123324869194195]
  unsupervised learning for superpixel segmentation via CNNs has been studied. We propose three key elements to improve the efficacy of such networks. By experimenting with the BSDS500 dataset, we find evidence to the significance of our proposal.
  arXiv  Detail & Related papers  (2022-06-21T09:30:26Z)
• Efficient Multiscale Object-based Superpixel Framework [62.48475585798724]
  We propose a novel superpixel framework, named Superpixels through Iterative CLEarcutting (SICLE) SICLE exploits object information being able to generate a multiscale segmentation on-the-fly. It generalizes recent superpixel methods, surpassing them and other state-of-the-art approaches in efficiency and effectiveness according to multiple delineation metrics.
  arXiv  Detail & Related papers  (2022-04-07T15:59:38Z)
• Saliency Enhancement using Superpixel Similarity [77.34726150561087]
  Saliency Object Detection (SOD) has several applications in image analysis. Deep-learning-based SOD methods are among the most effective, but they may miss foreground parts with similar colors. We introduce a post-processing method, named textitSaliency Enhancement over Superpixel Similarity (SESS) We demonstrate that SESS can consistently and considerably improve the results of three deep-learning-based SOD methods on five image datasets.
  arXiv  Detail & Related papers  (2021-12-01T17:22:54Z)
• OSLO: On-the-Sphere Learning for Omnidirectional images and its application to 360-degree image compression [59.58879331876508]
  We study the learning of representation models for omnidirectional images and propose to use the properties of HEALPix uniform sampling of the sphere to redefine the mathematical tools used in deep learning models for omnidirectional images. Our proposed on-the-sphere solution leads to a better compression gain that can save 13.7% of the bit rate compared to similar learned models applied to equirectangular images.
  arXiv  Detail & Related papers  (2021-07-19T22:14:30Z)
• Deep Superpixel Cut for Unsupervised Image Segmentation [0.9281671380673306]
  We propose a deep unsupervised method for image segmentation, which contains the following two stages. First, a Superpixelwise Autoencoder (SuperAE) is designed to learn the deep embedding and reconstruct a smoothed image, then the smoothed image is passed to generate superpixels. Second, we present a novel clustering algorithm called Deep Superpixel Cut (DSC), which measures the deep similarity between superpixels and formulates image segmentation as a soft partitioning problem.
  arXiv  Detail & Related papers  (2021-03-10T13:07:41Z)
• Superpixel Segmentation Based on Spatially Constrained Subspace Clustering [57.76302397774641]
  We consider each representative region with independent semantic information as a subspace, and formulate superpixel segmentation as a subspace clustering problem. We show that a simple integration of superpixel segmentation with the conventional subspace clustering does not effectively work due to the spatial correlation of the pixels. We propose a novel convex locality-constrained subspace clustering model that is able to constrain the spatial adjacent pixels with similar attributes to be clustered into a superpixel.
  arXiv  Detail & Related papers  (2020-12-11T06:18:36Z)
• Generalized Shortest Path-based Superpixels for Accurate Segmentation of Spherical Images [2.4063592468412267]
  We introduce a new superpixel method for spherical images called SphSPS (for Spherical Shortest Path-based Superpixels) Our approach respects the spherical geometry and generalizes the notion of shortest path between a pixel and a superpixel center on the 3D spherical acquisition space. We show that the feature information on such path can be efficiently integrated into our clustering framework and jointly improves the respect of object contours and the shape regularity.
  arXiv  Detail & Related papers  (2020-04-15T23:41:32Z)
• Superpixel Segmentation via Convolutional Neural Networks with Regularized Information Maximization [11.696069523681178]
  We propose an unsupervised superpixel segmentation method by optimizing a randomly-d convolutional neural network (CNN) in inference time. Our method generates superpixels via CNN from a single image without any labels by minimizing a proposed objective function for superpixel segmentation in inference time.
  arXiv  Detail & Related papers  (2020-02-17T04:32:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.