SILICA: Repurposing Diffusion Priors for Joint Glass Segmentation and Depth Estimation

• URL: http://arxiv.org/abs/2607.24249v1
• Date: Mon, 27 Jul 2026 10:38:49 GMT
• Title: SILICA: Repurposing Diffusion Priors for Joint Glass Segmentation and Depth Estimation
• Abstract summary: Standard depth sensors systematically fail on transparent surfaces, creating corrupted 3D maps and navigation hazards.<n>We introduce SILICA, a unified pipeline leveraging text-to-image diffusion models to jointly predict glass segmentation and glass-aware depth.<n> SILICA achieves remarkable zero-shot transfer across diverse, unseen environments, outperforming state-of-the-art models by almost 20%.
• Score: 11.131505915750013
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Standard depth sensors systematically fail on transparent surfaces, creating corrupted 3D maps and severe navigation hazards. While specialized hardware sensors can detect glass, they lack modularity and have extensive hardware dependencies. Consequently, learning-based monocular depth estimation has emerged as a compelling alternative. However, domain-specific glass-aware monocular depth estimators struggle with unfamiliar indoor layouts; restricted by the severe scarcity of real-world glass depth annotations, they fail to generalize zero-shot to new settings. This motivates us to explore whether the extensive priors of text-to-image diffusion models can enable generalizable perception of transparent surfaces. We introduce SILICA, a unified pipeline leveraging these priors to jointly predict glass segmentation and glass-aware depth. This mutual information exchange establishes a robust spatial hierarchy, entirely eliminating the need for paired real-world glass depth annotations. Subsequently, we use the predicted segmentation mask to explicitly filter incorrect glass depth points from standard sensors, recovering accurate metric glass depth for downstream 3D mapping and autonomous collision avoidance. Supported by our novel Mirage 18k dataset, extensive experiments demonstrate that SILICA achieves remarkable zero-shot transfer across diverse, unseen environments, outperforming state-of-the-art models by almost 20% and setting a new benchmark for transparent surface perception.

Related papers

• Glass Surface Detection Grounded in 3D Visual Geometry [118.92003088760919]
  Glass surface detection (GSD) is critical for scene understanding and reconstruction.<n>We propose a paradigm shift: grounding GSD in 3D visual geometry to explicitly model the physical existence of glass surfaces.<n>Our method first distills rich 3D priors from the visual geometry grounded transformer (VGGT) and generates glass-aware 3D representations.
  arXiv  Detail & Related papers  (2026-08-27T07:49:21Z)
• UniV2D: Bridging Visual Restoration and Semantic Perception for Underwater Salient Object Detection [71.83097731030254]
  We propose a Unified Vision-to-Detection Network (UniV2D) to jointly optimize visual restoration and salient object detection.<n>UniV2D introduces a semantic-driven learning paradigm: high-level saliency semantics actively guide the restoration process.<n>It significantly outperforms state-of-the-art methods in both quantitative and qualitative evaluations.
  arXiv  Detail & Related papers  (2026-05-08T02:31:45Z)
• Enhancing Glass Surface Reconstruction via Depth Prior for Robot Navigation [3.49604158786247]
  We propose a training-free framework that leverages depth foundation models as a structural prior.<n>We also introduce tiGlassRecon, a novel RGB-D dataset with geometrically derived ground truth for glass regions.
  arXiv  Detail & Related papers  (2026-04-20T14:35:31Z)
• Decompositional Neural Scene Reconstruction with Generative Diffusion Prior [64.71091831762214]
  Decompositional reconstruction of 3D scenes, with complete shapes and detailed texture, is intriguing for downstream applications.<n>Recent approaches incorporate semantic or geometric regularization to address this issue, but they suffer significant degradation in underconstrained areas.<n>We propose DP-Recon, which employs diffusion priors in the form of Score Distillation Sampling (SDS) to optimize the neural representation of each individual object under novel views.
  arXiv  Detail & Related papers  (2025-03-19T02:11:31Z)
• Revisiting Monocular 3D Object Detection with Depth Thickness Field [44.4805861813093]
  We present MonoDTF, a scene-to-instance depth-adapted network for monocular 3D object detection.<n>The framework mainly comprises a Scene-Level Depth Retargeting (SDR) module and an Instance-Level Spatial Refinement (ISR) module.<n>The latter refines the voxel space with the guidance of instances, enhancing the 3D instance-aware capability of the depth thickness field.
  arXiv  Detail & Related papers  (2024-12-26T10:51:50Z)
• Depth-aware Volume Attention for Texture-less Stereo Matching [67.46404479356896]
  We propose a lightweight volume refinement scheme to tackle the texture deterioration in practical outdoor scenarios. We introduce a depth volume supervised by the ground-truth depth map, capturing the relative hierarchy of image texture. Local fine structure and context are emphasized to mitigate ambiguity and redundancy during volume aggregation.
  arXiv  Detail & Related papers  (2024-02-14T04:07:44Z)
• Glass Segmentation with Multi Scales and Primary Prediction Guiding [2.66512000865131]
  Glass-like objects can be seen everywhere in our daily life which are hard for existing methods to segment them. We propose MGNet, which consists of a FineRescaling and Merging module (FRM) to improve the ability to extract semantics. We supervise the model with a novel loss function with the uncertainty-aware loss to produce high-confidence segmentation maps.
  arXiv  Detail & Related papers  (2024-02-13T16:14:32Z)
• MonoTDP: Twin Depth Perception for Monocular 3D Object Detection in Adverse Scenes [49.21187418886508]
  This paper proposes a monocular 3D detection model designed to perceive twin depth in adverse scenes, termed MonoTDP. We first introduce an adaptive learning strategy to aid the model in handling uncontrollable weather conditions, significantly resisting degradation caused by various degrading factors. Then, to address the depth/content loss in adverse regions, we propose a novel twin depth perception module that simultaneously estimates scene and object depth.
  arXiv  Detail & Related papers  (2023-05-18T13:42:02Z)
• On the Importance of Accurate Geometry Data for Dense 3D Vision Tasks [61.74608497496841]
  Training on inaccurate or corrupt data induces model bias and hampers generalisation capabilities. This paper investigates the effect of sensor errors for the dense 3D vision tasks of depth estimation and reconstruction.
  arXiv  Detail & Related papers  (2023-03-26T22:32:44Z)
• Leveraging RGB-D Data with Cross-Modal Context Mining for Glass Surface Detection [47.87834602551456]
  Glass surfaces are becoming increasingly ubiquitous as modern buildings tend to use a lot of glass panels.<n>This poses substantial challenges to the operations of autonomous systems such as robots, self-driving cars, and drones.<n>We propose a novel glass surface detection framework combining RGB and depth information.
  arXiv  Detail & Related papers  (2022-06-22T17:56:09Z)
• GlassNet: Label Decoupling-based Three-stream Neural Network for Robust Image Glass Detection [1.1825946875790057]
  We exploit label decoupling to decompose the labeled ground-truth (GT) map into an interior-diffusion map and a boundary-diffusion map. The GT map in collaboration with the two newly generated maps breaks the imbalanced distribution of the object boundary, leading to improved glass detection quality. We develop an attention-based boundary-aware feature Mosaic module to integrate multi-modal information.
  arXiv  Detail & Related papers  (2021-08-25T08:33:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.