Fixed External Cameras as Common Prior Maps for Active 3D Scene Graph Generation

• URL: http://arxiv.org/abs/2605.18184v1
• Date: Mon, 18 May 2026 10:26:07 GMT
• Title: Fixed External Cameras as Common Prior Maps for Active 3D Scene Graph Generation
• Abstract summary: We present an RGB-only framework for active, incremental 3D scene graph generation that seamlessly fuses observations from both onboard robot cameras and fixed external cameras within a single hardware-agnostic pipeline.<n> Experiments demonstrate that bootstrapping the scene graph with even a single external camera increases initial object recall by up to +79%, and that the richer context of the prior significantly improves the efficiency of subsequent active exploration.
• Score: 11.977612002830725
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Commonly available prior information, such as BIM models, floor plans, and remote sensing images, can provide valuable geometric and semantic context for autonomous robotic systems. In this paper, we treat observations from fixed external RGB cameras as Common Prior Maps (CPMs): wide-field views of the environment that initialize a semantic and geometric scene prior before any robot motion begins. We present an RGB-only framework for active, incremental 3D scene graph (3DSG) generation that seamlessly fuses observations from both onboard robot cameras and fixed external cameras within a single hardware-agnostic pipeline. By relying solely on RGB observations processed by a feed-forward 3D reconstruction model, the system treats all cameras - onboard or external - identically, requiring no hardware modifications. A graph-based active semantic exploration framework then directly leverages the partial scene graph to guide the robot toward regions of high semantic uncertainty, progressively completing and refining the prior. Experiments demonstrate that bootstrapping the scene graph with even a single external camera increases initial object recall by up to +79%, and that the richer context of the prior significantly improves the efficiency of subsequent active exploration.

Related papers

• See like a Robot: Robot-Centric Pointmaps for Vision-Language-Action Models [51.019904314336124]
  Vision-language-action (VLA) models predict robot actions from visual observations and language instructions.<n>Most VLAs observe the scene in the camera frame, creating a frame mismatch between where the scene is observed and where actions are defined.<n>We address this mismatch with robot-centric pointmaps, images whose pixels store the 3D coordinates of scene points in the robot frame.
  arXiv  Detail & Related papers  (2026-07-13T12:50:32Z)
• RGB-only Active 3D Scene Graph Generation for Indoor Mobile Robots [11.977612002830725]
  Current approaches to 3D scene graph generation rely on dedicated depth sensors, such as LiDAR or RGB-D cameras, for metric 3D reconstruction.<n>This paper presents a fully visual framework for the active, incremental construction of 3D scene graphs from RGB input only.
  arXiv  Detail & Related papers  (2026-05-18T10:37:39Z)
• FunRec: Reconstructing Functional 3D Scenes from Egocentric Interaction Videos [134.8812117202239]
  We present FunRec, a method for reconstructing functional 3D digital twins of indoor scenes directly from egocentric RGB-D interaction videos.<n>We demonstrate applications on URDF/USD export for simulation, hand-guided affordance mapping and robot-scene interaction.
  arXiv  Detail & Related papers  (2026-04-07T09:22:12Z)
• Robo3R: Enhancing Robotic Manipulation with Accurate Feed-Forward 3D Reconstruction [70.06600045165905]
  3D spatial perception is fundamental to general robotic manipulation, yet obtaining reliable, high-quality 3D geometry remains challenging.<n>We introduce Robo3R, a manipulation-ready 3D reconstruction model that predicts accurate, metric-scale scene geometry directly from RGB images and robot states in real time.<n>We observe consistent gains in performance, suggesting the promise of this alternative 3D sensing module for robotic manipulation.
  arXiv  Detail & Related papers  (2026-02-10T18:58:15Z)
• ArtiSG: Functional 3D Scene Graph Construction via Human-demonstrated Articulated Objects Manipulation [51.54082859171464]
  ArtiSG is a framework that constructs functional 3D scene graphs by encoding human demonstrations into structured robotic memory.<n>We show that ArtiSG significantly outperforms baselines in functional element recall and articulation estimation precision.
  arXiv  Detail & Related papers  (2025-12-31T13:10:40Z)
• Unifying Scene Representation and Hand-Eye Calibration with 3D Foundation Models [13.58353565350936]
  Representing the environment is a central challenge in robotics. Traditionally, users need to calibrate the camera using a specific external marker, such as a checkerboard or AprilTag. This paper advocates for the integration of 3D foundation representation into robotic systems equipped with manipulator-mounted RGB cameras.
  arXiv  Detail & Related papers  (2024-04-17T18:29:32Z)
• ImageManip: Image-based Robotic Manipulation with Affordance-guided Next View Selection [10.162882793554191]
  3D articulated object manipulation is essential for enabling robots to interact with their environment. Many existing studies make use of 3D point clouds as the primary input for manipulation policies. RGB images offer high-resolution observations using cost effective devices but lack spatial 3D geometric information. This framework is designed to capture multiple perspectives of the target object and infer depth information to complement its geometry.
  arXiv  Detail & Related papers  (2023-10-13T12:42:54Z)
• Event Camera-based Visual Odometry for Dynamic Motion Tracking of a Legged Robot Using Adaptive Time Surface [5.341864681049579]
  Event cameras offer high temporal resolution and dynamic range, which can eliminate the issue of blurred RGB images during fast movements. We introduce an adaptive time surface (ATS) method that addresses the whiteout and blackout issue in conventional time surfaces. Lastly, we propose a nonlinear pose optimization formula that simultaneously performs 3D-2D alignment on both RGB-based and event-based maps and images.
  arXiv  Detail & Related papers  (2023-05-15T19:03:45Z)
• RGB-Only Reconstruction of Tabletop Scenes for Collision-Free Manipulator Control [71.51781695764872]
  We present a system for collision-free control of a robot manipulator that uses only RGB views of the world. Perceptual input of a tabletop scene is provided by multiple images of an RGB camera that is either handheld or mounted on the robot end effector. A NeRF-like process is used to reconstruct the 3D geometry of the scene, from which the Euclidean full signed distance function (ESDF) is computed. A model predictive control algorithm is then used to control the manipulator to reach a desired pose while avoiding obstacles in the ESDF.
  arXiv  Detail & Related papers  (2022-10-21T01:45:08Z)
• Neural Scene Representation for Locomotion on Structured Terrain [56.48607865960868]
  We propose a learning-based method to reconstruct the local terrain for a mobile robot traversing urban environments. Using a stream of depth measurements from the onboard cameras and the robot's trajectory, the estimates the topography in the robot's vicinity. We propose a 3D reconstruction model that faithfully reconstructs the scene, despite the noisy measurements and large amounts of missing data coming from the blind spots of the camera arrangement.
  arXiv  Detail & Related papers  (2022-06-16T10:45:17Z)
• Real-time RGBD-based Extended Body Pose Estimation [57.61868412206493]
  We present a system for real-time RGBD-based estimation of 3D human pose. We use parametric 3D deformable human mesh model (SMPL-X) as a representation. We train estimators of body pose and facial expression parameters.
  arXiv  Detail & Related papers  (2021-03-05T13:37:50Z)
• RGB-D Odometry and SLAM [20.02647320786556]
  RGB-D sensors are low-cost, low-power and low-size alternatives to traditional range sensors such as LiDAR. Unlike RGB cameras, RGB-D sensors provide the additional depth information that removes the need of frame-by-frame triangulation for 3D scene reconstruction. This chapter consists of three main parts: In the first part, we introduce the basic concept of odometry and SLAM and motivate the use of RGB-D sensors. In the second part, we detail the three main components of SLAM systems: camera pose tracking, scene mapping and loop closing.
  arXiv  Detail & Related papers  (2020-01-19T17:56:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.