RayLift: Lifting Complementary Ray-Wise Evidence with 3D Geometry Priors for Semantic Scene Completion

• URL: http://arxiv.org/abs/2608.08476v1
• Date: Sun, 09 Aug 2026 04:41:34 GMT
• Title: RayLift: Lifting Complementary Ray-Wise Evidence with 3D Geometry Priors for Semantic Scene Completion
• Abstract summary: Camera-based 3D semantic scene completion (SSC) provides comprehensive scene understanding for autonomous driving and robotics.<n>Existing methods often treat stereo depth estimates as deterministic geometric constraints, causing depth uncertainty and local correspondence errors to propagate directly into voxel representations.<n>We propose RayLift, a framework that uses stereo geometry as a metric reference while incorporating complementary ray evidence to recover reliable 3D structures adaptively.
• Score: 23.7438085195588
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Camera-based 3D semantic scene completion (SSC) provides comprehensive scene understanding for autonomous driving and robotics. However, existing methods often treat stereo depth estimates as deterministic geometric constraints, causing depth uncertainty and local correspondence errors to propagate directly into voxel representations. To address this issue, we propose RayLift, a framework that uses stereo geometry as a metric reference while incorporating complementary ray evidence to recover reliable 3D structures adaptively. RayLift first employs a Complementary Context Encoder that extracts geometry-aware priors from a frozen 3D vision foundation model, thereby enriching the scene context. It then introduces a Depth Ray Evidence Lifter module that jointly models geometric dissimilarity, depth confidence, and spatial uncertainty to adaptively sample and weight candidate surface locations along each camera ray. Finally, a Semantic-Aware Voxel Integrator injects the resulting ray evidence into voxel features by explicitly modeling their spatial support. Extensive experiments on SemanticKITTI and SSCBench-KITTI-360 demonstrate that RayLift achieves competitive performance and consistently outperforms existing methods.

Related papers

• RaysUp: Ultra-light Universal Feature Upsampling via Geometry-Aware Ray Representation [11.94699228168484]
  RaysUp is an ultra-lightweight, task-agnostic, and VFM-agnostic feature upsampling framework.<n>It reconstructs high-resolution feature maps at arbitrary dense generalizations.<n>Experiments demonstrate that RaysUp state-of-the-art performance while using only 16% of the parameters of AnyUp.
  arXiv  Detail & Related papers  (2026-06-22T01:37:56Z)
• 3D Consistency Optimization for Self-Supervised Monocular Video Depth Estimation [21.287558280133076]
  We introduce a new paradigm that recasts sequential video depth estimation as an unconstrained multi-view 3D reconstruction problem.<n>Our approach is driven by three constraints: image-level photometric rendering, explicit world-coordinate geometric alignment, and multi-scale temporal gradient consistency.<n>Results show that the proposed approach achieves state-of-the-art spatial accuracy, outperforming the frame-based, video-based depth estimators and the multi-view 3D reconstruction baselines.
  arXiv  Detail & Related papers  (2026-06-14T08:56:45Z)
• ViPS: Video-informed Pose Spaces for Auto-Rigged Meshes [55.32681167870698]
  Video-informed Pose Spaces (ViPS) is a feed-forward framework that discovers the latent distribution of valid articulations for auto-rigged meshes.<n>ViPS transfers generative video priors into a universal distribution over a given rig parameterization.<n>Our evaluations show that ViPS matches the performance of state-of-the-art methods trained on synthetic artist-created 4D data in both plausibility and diversity.
  arXiv  Detail & Related papers  (2026-04-19T21:21:11Z)
• Think, Act, Build: An Agentic Framework with Vision Language Models for Zero-Shot 3D Visual Grounding [34.1504914582344]
  3D Visual Grounding aims to localize objects in 3D scenes via natural language descriptions.<n>We propose "Think, Act, Build (TAB)", a generative 2D-to-3D reconstruction paradigm operating directly on raw RGB-D streams.<n>To overcome the multi-view coverage deficit caused by strict VLM semantic tracking, we introduce the Semantic-Anchored Geometric Expansion.
  arXiv  Detail & Related papers  (2026-04-01T06:12:16Z)
• SpatialFly: Geometry-Guided Representation Alignment for UAV Vision-and-Language Navigation in Urban Environments [49.966170814478915]
  We propose a geometry-guided spatial representation framework for UAV VLN.<n>Running on RGB observations without explicit 3D reconstruction, SpatialFly introduces a geometry-guided 2D representation alignment mechanism.<n> Experimental results show that SpatialFly consistently outperforms state-of-the-art UAV VLN baselines across both seen and unseen environments.
  arXiv  Detail & Related papers  (2026-03-22T03:56:58Z)
• Generation Models Know Space: Unleashing Implicit 3D Priors for Scene Understanding [50.098085774845195]
  We propose a paradigm shift by leveraging the implicit spatial prior within large-scale generation models.<n>We introduce GeneEGA-3D (Video Extracted Generative Awareness), a plug-and-play framework that repurposes a pre-trained video diffusion model as a Latent World Simulator.
  arXiv  Detail & Related papers  (2026-03-19T17:59:58Z)
• GLaD: Geometric Latent Distillation for Vision-Language-Action Models [106.53332923530245]
  We introduce GLaD, a geometry-aware Vision-Language-Action (VLA) framework that incorporates 3D geometric priors during pretraining through knowledge distillation.<n>GLaD achieves 94.1% average success rate across four LIBERO task suites, outperforming UniVLA (92.5%) which uses identical pretraining data.
  arXiv  Detail & Related papers  (2025-12-10T13:07:27Z)
• TALO: Pushing 3D Vision Foundation Models Towards Globally Consistent Online Reconstruction [57.46712611558817]
  3D vision foundation models have shown strong generalization in reconstructing key 3D attributes from uncalibrated images through a single feed-forward pass.<n>Recent strategies align consecutive predictions by solving global transformation, yet our analysis reveals their fundamental limitations in assumption validity, local alignment scope, and robustness under noisy geometry.<n>We propose a higher-DOF and long-term alignment framework based on Thin Plate Spline, leveraging globally propagated control points to correct spatially varying inconsistencies.
  arXiv  Detail & Related papers  (2025-12-02T02:22:20Z)
• PLANA3R: Zero-shot Metric Planar 3D Reconstruction via Feed-Forward Planar Splatting [56.188624157291024]
  We introduce PLANA3R, a pose-free framework for metric Planar 3D Reconstruction from unposed two-view images.<n>Unlike prior feedforward methods that require 3D plane annotations during training, PLANA3R learns planar 3D structures without explicit plane supervision.<n>We validate PLANA3R on multiple indoor-scene datasets with metric supervision and demonstrate strong generalization to out-of-domain indoor environments.
  arXiv  Detail & Related papers  (2025-10-21T15:15:33Z)
• ViPOcc: Leveraging Visual Priors from Vision Foundation Models for Single-View 3D Occupancy Prediction [11.312780421161204]
  In this paper, we propose ViPOcc, which leverages the visual priors from vision foundation models for fine-grained 3D occupancy prediction.<n>We also propose a semantic-guided non-overlapping Gaussian mixture sampler for efficient, instance-aware ray sampling.<n>Our experiments demonstrate the superior performance of ViPOcc in both 3D occupancy prediction and depth estimation tasks.
  arXiv  Detail & Related papers  (2024-12-15T15:04:27Z)
• Bridging Stereo Geometry and BEV Representation with Reliable Mutual Interaction for Semantic Scene Completion [45.171150395915056]
  3D semantic scene completion (SSC) is an ill-posed perception task that requires inferring a dense 3D scene from limited observations. Previous camera-based methods struggle to predict accurate semantic scenes due to inherent geometric ambiguity and incomplete observations. We resort to stereo matching technique and bird's-eye-view (BEV) representation learning to address such issues in SSC.
  arXiv  Detail & Related papers  (2023-03-24T12:33:44Z)
• Joint stereo 3D object detection and implicit surface reconstruction [39.30458073540617]
  We present a new learning-based framework S-3D-RCNN that can recover accurate object orientation in SO(3) and simultaneously predict implicit rigid shapes from stereo RGB images. For orientation estimation, in contrast to previous studies that map local appearance to observation angles, we propose a progressive approach by extracting meaningful Intermediate Geometrical Representations (IGRs) This approach features a deep model that transforms perceived intensities from one or two views to object part coordinates to achieve direct egocentric object orientation estimation in the camera coordinate system. To further achieve finer description inside 3D bounding boxes, we investigate the implicit shape estimation problem from stereo images
  arXiv  Detail & Related papers  (2021-11-25T05:52:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.