GeomPrompt: Geometric Prompt Learning for RGB-D Semantic Segmentation Under Missing and Degraded Depth

• URL: http://arxiv.org/abs/2604.11585v1
• Date: Mon, 13 Apr 2026 15:01:22 GMT
• Title: GeomPrompt: Geometric Prompt Learning for RGB-D Semantic Segmentation Under Missing and Degraded Depth
• Abstract summary: GeomPrompt is a lightweight adaptation module that synthesizes a task-driven geometric prompt from RGB alone.<n>GeomPrompt-Recovery compensates for degraded depth by predicting the fourth channel correction relevant for the frozen segmenter.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal perception systems for robotics and embodied AI often assume reliable RGB-D sensing, but in practice, depth is frequently missing, noisy, or corrupted. We thus present GeomPrompt, a lightweight cross-modal adaptation module that synthesizes a task-driven geometric prompt from RGB alone for the fourth channel of a frozen RGB-D semantic segmentation model, without depth supervision. We further introduce GeomPrompt-Recovery, an adaptation module that compensates for degraded depth by predicting the fourth channel correction relevant for the frozen segmenter. Both modules are trained solely with downstream segmentation supervision, enabling recovery of the geometric prior useful for segmentation, rather than estimating depth signals. On SUN RGB-D, GeomPrompt improves over RGB-only inference by +6.1 mIoU on DFormer and +3.0 mIoU on GeminiFusion, while remaining competitive with strong monocular depth estimators. For degraded depth, GeomPrompt-Recovery consistently improves robustness, yielding gains up to +3.6 mIoU under severe depth corruptions. GeomPrompt is also substantially more efficient than monocular depth baselines, reaching 7.8 ms latency versus 38.3 ms and 71.9 ms. These results suggest that task-driven geometric prompting is an efficient mechanism for cross-modal compensation under missing and degraded depth inputs in RGB-D perception.

Related papers

• RIDE: Relocalization-Informed Depth Estimation with 3D Gaussian Splatting [20.29273985020268]
  We present RIDE, which estimates dense depth from a robot's RGB map.<n>Experiments show improved depth accuracy temporal consistency scale-only calibration, demonstrating how dense geometry can support both pose recovery and robot perception.
  arXiv  Detail & Related papers  (2026-09-10T04:36:58Z)
• When Depth Hurts: Reliability-Aware Geometry Distillation for Depth-Free RGB-D Salient Object Detection [39.23633029110609]
  Missing regions, blurred boundaries, and structural artifacts can propagate through multimodal fusion and make an RGB-D detector less accurate than its RGB-only counterpart.<n>We propose method, a reliability-aware geometry distillation framework developed for RGB-D SOD benchmarks without using dataset-provided depth during training or inference.<n>Method achieves the best or tied-best result in 26 of 36 metric-dataset comparisons against ten recent RGB-D SOD methods, including a 13.4% relative MAE reduction on ReDWeb-S.
  arXiv  Detail & Related papers  (2026-09-03T05:29:10Z)
• EfficientPENet: Real-Time Depth Completion from Sparse LiDAR via Lightweight Multi-Modal Fusion [1.9903316442426757]
  Existing methods achieve high accuracy on standard benchmarks but rely on heavy backbone architectures that preclude real-time deployment on embedded hardware.<n>We present EfficientPENet, a two-branch depth completion network that replaces the conventional ResNet encoder with a modernized ConvNeXt backbone.<n>On the KITTI depth completion benchmark, EfficientPENet achieves an RMSE of 631.94 mm with 36.24M parameters and a latency of 20.51 ms, operating at 48.76 FPS.
  arXiv  Detail & Related papers  (2026-04-20T19:56:04Z)
• PDDM: Pseudo Depth Diffusion Model for RGB-PD Semantic Segmentation Based in Complex Indoor Scenes [6.698379291727345]
  Pseudo Depth (PD) from high-precision depth estimation algorithms can eliminate the dependence on RGB-D sensors and alignment processes.<n>PD shows significant potential in semantic segmentation.<n>PD aggregates multiple pseudo depth maps into a single modality.<n>PD achieves state-of-the-art performance, outperforming other methods by +6.98 mIoU on NYUv2 and +2.11 mIoU on SUNRGB-D.
  arXiv  Detail & Related papers  (2025-03-24T07:05:31Z)
• Depth Matters: Exploring Deep Interactions of RGB-D for Semantic Segmentation in Traffic Scenes [11.446541235218396]
  We propose a novel learnable Depth interaction Pyramid Transformer (DiPFormer) to explore the effectiveness of depth.<n>DiPFormer achieves state-of-the-art performance on the KITTI (97.57% F-score on KITTI road and 68.74% mIoU on KITTI-360) and Cityscapes datasets.
  arXiv  Detail & Related papers  (2024-09-12T12:39:34Z)
• Symmetric Uncertainty-Aware Feature Transmission for Depth Super-Resolution [52.582632746409665]
  We propose a novel Symmetric Uncertainty-aware Feature Transmission (SUFT) for color-guided DSR. Our method achieves superior performance compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-06-01T06:35:59Z)
• RGB-D Grasp Detection via Depth Guided Learning with Cross-modal Attention [14.790193023912973]
  This paper proposes a novel learning based approach to RGB-D grasp detection, namely Depth Guided Cross-modal Attention Network (DGCAN) To better leverage the geometry information recorded in the depth channel, a complete 6-dimensional rectangle representation is adopted with the grasp depth dedicatedly considered. The prediction of the extra grasp depth substantially strengthens feature learning, thereby leading to more accurate results.
  arXiv  Detail & Related papers  (2023-02-28T02:41:27Z)
• Dual Swin-Transformer based Mutual Interactive Network for RGB-D Salient Object Detection [67.33924278729903]
  In this work, we propose Dual Swin-Transformer based Mutual Interactive Network. We adopt Swin-Transformer as the feature extractor for both RGB and depth modality to model the long-range dependencies in visual inputs. Comprehensive experiments on five standard RGB-D SOD benchmark datasets demonstrate the superiority of the proposed DTMINet method.
  arXiv  Detail & Related papers  (2022-06-07T08:35:41Z)
• Cross-modality Discrepant Interaction Network for RGB-D Salient Object Detection [78.47767202232298]
  We propose a novel Cross-modality Discrepant Interaction Network (CDINet) for RGB-D SOD. Two components are designed to implement the effective cross-modality interaction. Our network outperforms $15$ state-of-the-art methods both quantitatively and qualitatively.
  arXiv  Detail & Related papers  (2021-08-04T11:24:42Z)
• Deep RGB-D Saliency Detection with Depth-Sensitive Attention and Automatic Multi-Modal Fusion [15.033234579900657]
  RGB-D salient object detection (SOD) is usually formulated as a problem of classification or regression over two modalities, i.e., RGB and depth. We propose a depth-sensitive RGB feature modeling scheme using the depth-wise geometric prior of salient objects. Experiments on seven standard benchmarks demonstrate the effectiveness of the proposed approach against the state-of-the-art.
  arXiv  Detail & Related papers  (2021-03-22T13:28:45Z)
• Progressively Guided Alternate Refinement Network for RGB-D Salient Object Detection [63.18846475183332]
  We aim to develop an efficient and compact deep network for RGB-D salient object detection. We propose a progressively guided alternate refinement network to refine it. Our model outperforms existing state-of-the-art approaches by a large margin.
  arXiv  Detail & Related papers  (2020-08-17T02:55:06Z)
• Data-Level Recombination and Lightweight Fusion Scheme for RGB-D Salient Object Detection [73.31632581915201]
  We propose a novel data-level recombination strategy to fuse RGB with D (depth) before deep feature extraction. A newly lightweight designed triple-stream network is applied over these novel formulated data to achieve an optimal channel-wise complementary fusion status between the RGB and D.
  arXiv  Detail & Related papers  (2020-08-07T10:13:05Z)
• Cross-Modal Weighting Network for RGB-D Salient Object Detection [76.0965123893641]
  We propose a novel Cross-Modal Weighting (CMW) strategy to encourage comprehensive interactions between RGB and depth channels for RGB-D SOD. Specifically, three RGB-depth interaction modules, named CMW-L, CMW-M and CMW-H, are developed to deal with respectively low-, middle- and high-level cross-modal information fusion. CMWNet consistently outperforms 15 state-of-the-art RGB-D SOD methods on seven popular benchmarks.
  arXiv  Detail & Related papers  (2020-07-09T16:01:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.