IAM: Enhancing RGB-D Instance Segmentation with New Benchmarks

• URL: http://arxiv.org/abs/2501.01685v1
• Date: Fri, 03 Jan 2025 08:03:24 GMT
• Title: IAM: Enhancing RGB-D Instance Segmentation with New Benchmarks
• Authors: Aecheon Jung, Soyun Choi, Junhong Min, Sungeun Hong,
• Abstract summary: RGB-D segmentation promises richer scene understanding than RGB-only methods. There is a relative scarcity of instance-level RGB-D segmentation datasets. We introduce three RGB-D instance segmentation benchmarks, distinguished at the instance level. We propose a simple yet effective method for RGB-D data integration.
• Score: 4.3266254914862445
• License:
• Abstract: Image segmentation is a vital task for providing human assistance and enhancing autonomy in our daily lives. In particular, RGB-D segmentation-leveraging both visual and depth cues-has attracted increasing attention as it promises richer scene understanding than RGB-only methods. However, most existing efforts have primarily focused on semantic segmentation and thus leave a critical gap. There is a relative scarcity of instance-level RGB-D segmentation datasets, which restricts current methods to broad category distinctions rather than fully capturing the fine-grained details required for recognizing individual objects. To bridge this gap, we introduce three RGB-D instance segmentation benchmarks, distinguished at the instance level. These datasets are versatile, supporting a wide range of applications from indoor navigation to robotic manipulation. In addition, we present an extensive evaluation of various baseline models on these benchmarks. This comprehensive analysis identifies both their strengths and shortcomings, guiding future work toward more robust, generalizable solutions. Finally, we propose a simple yet effective method for RGB-D data integration. Extensive evaluations affirm the effectiveness of our approach, offering a robust framework for advancing toward more nuanced scene understanding.

Related papers

• Diffusion-based RGB-D Semantic Segmentation with Deformable Attention Transformer [10.982521876026281]
  We introduce a diffusion-based framework to address the RGB-D semantic segmentation problem. We demonstrate that utilizing a Deformable Attention Transformer as the encoder to extract features from depth images effectively captures the characteristics of invalid regions in depth measurements.
  arXiv  Detail & Related papers  (2024-09-23T15:23:01Z)
• HS3-Bench: A Benchmark and Strong Baseline for Hyperspectral Semantic Segmentation in Driving Scenarios [3.7498611358320733]
  There is no standard benchmark available to measure progress on semantic segmentation in driving scenarios. In this paper, we provide the HyperSpectral Semantic benchmark (HS3-Bench) It combines annotated hyperspectral images from three driving scenario datasets and provides standardized metrics, implementations, and evaluation protocols.
  arXiv  Detail & Related papers  (2024-09-17T14:00:49Z)
• Segment Any Events via Weighted Adaptation of Pivotal Tokens [85.39087004253163]
  This paper focuses on the nuanced challenge of tailoring the Segment Anything Models (SAMs) for integration with event data. We introduce a multi-scale feature distillation methodology to optimize the alignment of token embeddings originating from event data with their RGB image counterparts.
  arXiv  Detail & Related papers  (2023-12-24T12:47:08Z)
• Spatial-information Guided Adaptive Context-aware Network for Efficient RGB-D Semantic Segmentation [9.198120596225968]
  We propose an efficient lightweight encoder-decoder network that reduces the computational parameters and guarantees the robustness of the algorithm. Experimental results on NYUv2, SUN RGB-D, and Cityscapes datasets show that our method achieves a better trade-off among segmentation accuracy, inference time, and parameters than the state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-08-11T09:02:03Z)
• A Threefold Review on Deep Semantic Segmentation: Efficiency-oriented, Temporal and Depth-aware design [77.34726150561087]
  We conduct a survey on the most relevant and recent advances in Deep Semantic in the context of vision for autonomous vehicles. Our main objective is to provide a comprehensive discussion on the main methods, advantages, limitations, results and challenges faced from each perspective.
  arXiv  Detail & Related papers  (2023-03-08T01:29:55Z)
• RGB-D Saliency Detection via Cascaded Mutual Information Minimization [122.8879596830581]
  Existing RGB-D saliency detection models do not explicitly encourage RGB and depth to achieve effective multi-modal learning. We introduce a novel multi-stage cascaded learning framework via mutual information minimization to "explicitly" model the multi-modal information between RGB image and depth data.
  arXiv  Detail & Related papers  (2021-09-15T12:31:27Z)
• Efficient RGB-D Semantic Segmentation for Indoor Scene Analysis [16.5390740005143]
  We propose an efficient and robust RGB-D segmentation approach that can be optimized to a high degree using NVIDIART. We show that RGB-D segmentation is superior to processing RGB images solely and that it can still be performed in real time if the network architecture is carefully designed.
  arXiv  Detail & Related papers  (2020-11-13T15:17:31Z)
• Bi-directional Cross-Modality Feature Propagation with Separation-and-Aggregation Gate for RGB-D Semantic Segmentation [59.94819184452694]
  Depth information has proven to be a useful cue in the semantic segmentation of RGBD images for providing a geometric counterpart to the RGB representation. Most existing works simply assume that depth measurements are accurate and well-aligned with the RGB pixels and models the problem as a cross-modal feature fusion. In this paper, we propose a unified and efficient Crossmodality Guided to not only effectively recalibrate RGB feature responses, but also to distill accurate depth information via multiple stages and aggregate the two recalibrated representations alternatively.
  arXiv  Detail & Related papers  (2020-07-17T18:35:24Z)
• Bifurcated backbone strategy for RGB-D salient object detection [168.19708737906618]
  We leverage the inherent multi-modal and multi-level nature of RGB-D salient object detection to devise a novel cascaded refinement network. Our architecture, named Bifurcated Backbone Strategy Network (BBS-Net), is simple, efficient, and backbone-independent.
  arXiv  Detail & Related papers  (2020-07-06T13:01:30Z)
• Is Depth Really Necessary for Salient Object Detection? [50.10888549190576]
  We make the first attempt in realizing an unified depth-aware framework with only RGB information as input for inference. Not only surpasses the state-of-the-art performances on five public RGB SOD benchmarks, but also surpasses the RGBD-based methods on five benchmarks by a large margin.
  arXiv  Detail & Related papers  (2020-05-30T13:40:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.