Exploiting Object-based and Segmentation-based Semantic Features for Deep Learning-based Indoor Scene Classification

• URL: http://arxiv.org/abs/2404.07739v1
• Date: Thu, 11 Apr 2024 13:37:51 GMT
• Title: Exploiting Object-based and Segmentation-based Semantic Features for Deep Learning-based Indoor Scene Classification
• Authors: Ricardo Pereira, Luís Garrote, Tiago Barros, Ana Lopes, Urbano J. Nunes,
• Abstract summary: The work described in this paper uses both semantic information, obtained from object detection, and semantic segmentation techniques. A novel approach that uses a semantic segmentation mask to provide Hu-moments-based segmentation categories' shape characterization, designated by Hu-Moments Features (SHMFs) is proposed. A three-main-branch network, designated by GOS$2$F$2$App, that exploits deep-learning-based global features, object-based features, and semantic segmentation-based features is also proposed.
• Score: 0.5572976467442564
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Indoor scenes are usually characterized by scattered objects and their relationships, which turns the indoor scene classification task into a challenging computer vision task. Despite the significant performance boost in classification tasks achieved in recent years, provided by the use of deep-learning-based methods, limitations such as inter-category ambiguity and intra-category variation have been holding back their performance. To overcome such issues, gathering semantic information has been shown to be a promising source of information towards a more complete and discriminative feature representation of indoor scenes. Therefore, the work described in this paper uses both semantic information, obtained from object detection, and semantic segmentation techniques. While object detection techniques provide the 2D location of objects allowing to obtain spatial distributions between objects, semantic segmentation techniques provide pixel-level information that allows to obtain, at a pixel-level, a spatial distribution and shape-related features of the segmentation categories. Hence, a novel approach that uses a semantic segmentation mask to provide Hu-moments-based segmentation categories' shape characterization, designated by Segmentation-based Hu-Moments Features (SHMFs), is proposed. Moreover, a three-main-branch network, designated by GOS$^2$F$^2$App, that exploits deep-learning-based global features, object-based features, and semantic segmentation-based features is also proposed. GOS$^2$F$^2$App was evaluated in two indoor scene benchmark datasets: SUN RGB-D and NYU Depth V2, where, to the best of our knowledge, state-of-the-art results were achieved on both datasets, which present evidences of the effectiveness of the proposed approach.

Related papers

• N2F2: Hierarchical Scene Understanding with Nested Neural Feature Fields [112.02885337510716]
  Nested Neural Feature Fields (N2F2) is a novel approach that employs hierarchical supervision to learn a single feature field. We leverage a 2D class-agnostic segmentation model to provide semantically meaningful pixel groupings at arbitrary scales in the image space. Our approach outperforms the state-of-the-art feature field distillation methods on tasks such as open-vocabulary 3D segmentation and localization.
  arXiv  Detail & Related papers  (2024-03-16T18:50:44Z)
• Inter-object Discriminative Graph Modeling for Indoor Scene Recognition [5.712940060321454]
  We propose to leverage discriminative object knowledge to enhance scene feature representations. We construct a Discriminative Graph Network (DGN) in which pixel-level scene features are defined as nodes. With the proposed IODP and DGN, we obtain state-of-the-art results on several widely used scene datasets.
  arXiv  Detail & Related papers  (2023-11-10T08:07:16Z)
• Weakly-supervised Contrastive Learning for Unsupervised Object Discovery [52.696041556640516]
  Unsupervised object discovery is promising due to its ability to discover objects in a generic manner. We design a semantic-guided self-supervised learning model to extract high-level semantic features from images. We introduce Principal Component Analysis (PCA) to localize object regions.
  arXiv  Detail & Related papers  (2023-07-07T04:03:48Z)
• A Deep Learning-based Global and Segmentation-based Semantic Feature Fusion Approach for Indoor Scene Classification [0.6578923037198714]
  This work proposes a novel approach that uses a semantic segmentation mask to obtain a 2D spatial layout of the segmentation-categories across the scene. A two-branch network, GS2F2App, exploits CNN-based global features extracted from RGB images and the segmentation-based features extracted from the proposed SSFs.
  arXiv  Detail & Related papers  (2023-02-13T15:12:11Z)
• Part-guided Relational Transformers for Fine-grained Visual Recognition [59.20531172172135]
  We propose a framework to learn the discriminative part features and explore correlations with a feature transformation module. Our proposed approach does not rely on additional part branches and reaches state-the-of-art performance on 3-of-the-level object recognition.
  arXiv  Detail & Related papers  (2022-12-28T03:45:56Z)
• SemAffiNet: Semantic-Affine Transformation for Point Cloud Segmentation [94.11915008006483]
  We propose SemAffiNet for point cloud semantic segmentation. We conduct extensive experiments on the ScanNetV2 and NYUv2 datasets.
  arXiv  Detail & Related papers  (2022-05-26T17:00:23Z)
• Deep ViT Features as Dense Visual Descriptors [12.83702462166513]
  We leverage deep features extracted from a pre-trained Vision Transformer (ViT) as dense visual descriptors. These descriptors facilitate a variety of applications, including co-segmentation, part co-segmentation and correspondences.
  arXiv  Detail & Related papers  (2021-12-10T20:15:03Z)
• Boundary Knowledge Translation based Reference Semantic Segmentation [62.60078935335371]
  We introduce a Reference Reference segmentation Network (Ref-Net) to conduct visual boundary knowledge translation. Inspired by the human recognition mechanism, RSMTM is devised only to segment the same category objects based on the features of the reference objects. With tens of finely-grained annotated samples as guidance, Ref-Net achieves results on par with fully supervised methods on six datasets.
  arXiv  Detail & Related papers  (2021-08-01T07:40:09Z)
• Unsupervised segmentation via semantic-apparent feature fusion [21.75371777263847]
  This research proposes an unsupervised foreground segmentation method based on semantic-apparent feature fusion (SAFF) Key regions of foreground object can be accurately responded via semantic features, while apparent features provide richer detailed expression. By fusing semantic and apparent features, as well as cascading the modules of intra-image adaptive feature weight learning and inter-image common feature learning, the research achieves performance that significantly exceeds baselines.
  arXiv  Detail & Related papers  (2020-05-21T08:28:49Z)
• SceneEncoder: Scene-Aware Semantic Segmentation of Point Clouds with A Learnable Scene Descriptor [51.298760338410624]
  We propose a SceneEncoder module to impose a scene-aware guidance to enhance the effect of global information. The module predicts a scene descriptor, which learns to represent the categories of objects existing in the scene. We also design a region similarity loss to propagate distinguishing features to their own neighboring points with the same label.
  arXiv  Detail & Related papers  (2020-01-24T16:53:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.