RobotPan: A 360$^\circ$ Surround-View Robotic Vision System for Embodied Perception

• URL: http://arxiv.org/abs/2604.13476v1
• Date: Wed, 15 Apr 2026 04:58:23 GMT
• Title: RobotPan: A 360$^\circ$ Surround-View Robotic Vision System for Embodied Perception
• Abstract summary: We introduce a surround-view robotic vision system that combines six cameras with LiDAR to provide full 360$circ$ visual coverage.<n>We also present textscRobotPan, a feed-forward framework that predicts emphmetric-scaled and emphcompact 3D Gaussians from calibrated sparse-view inputs.<n>Experiments show that textscRobotPan achieves competitive quality against prior feed-forward reconstruction and view-synthesis methods.
• Score: 47.76543396190029
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Surround-view perception is increasingly important for robotic navigation and loco-manipulation, especially in human-in-the-loop settings such as teleoperation, data collection, and emergency takeover. However, current robotic visual interfaces are often limited to narrow forward-facing views, or, when multiple on-board cameras are available, require cumbersome manual switching that interrupts the operator's workflow. Both configurations suffer from motion-induced jitter that causes simulator sickness in head-mounted displays. We introduce a surround-view robotic vision system that combines six cameras with LiDAR to provide full 360$^\circ$ visual coverage, while meeting the geometric and real-time constraints of embodied deployment. We further present \textsc{RobotPan}, a feed-forward framework that predicts \emph{metric-scaled} and \emph{compact} 3D Gaussians from calibrated sparse-view inputs for real-time rendering, reconstruction, and streaming. \textsc{RobotPan} lifts multi-view features into a unified spherical coordinate representation and decodes Gaussians using hierarchical spherical voxel priors, allocating fine resolution near the robot and coarser resolution at larger radii to reduce computational redundancy without sacrificing fidelity. To support long sequences, our online fusion updates dynamic content while preventing unbounded growth in static regions by selectively updating appearance. Finally, we release a multi-sensor dataset tailored to 360$^\circ$ novel view synthesis and metric 3D reconstruction for robotics, covering navigation, manipulation, and locomotion on real platforms. Experiments show that \textsc{RobotPan} achieves competitive quality against prior feed-forward reconstruction and view-synthesis methods while producing substantially fewer Gaussians, enabling practical real-time embodied deployment. Project website: https://robotpan.github.io/

Related papers

• Tele360: Real-Time Feed-Forward Human Reconstruction from Sparse Unposed Cameras [43.56045236607455]
  We present Tele360, the first real-time feed-forward system for dynamic human reconstruction and live free-viewpoint visualization.<n>Our system jointly estimates camera poses and reconstructs a dynamic 3D Gaussian representation for each time instance.<n>Experiments show that Tele360 achieves state-of-the-art visual quality on studio benchmarks while supporting real-time 2K input-to-rendering at over 25 FPS on a single consumer GPU.
  arXiv  Detail & Related papers  (2026-09-14T04:50:52Z)
• GRAIL: Generating Humanoid Loco-Manipulation from 3D Assets and Video Priors [113.71148915419246]
  We present GRAIL, a digital generation pipeline that composes 3D assets, simulator-ready scenes, and priors from video foundation models (VFMs) to synthesize interactions without rebuilding physical environments or teleoperating the robot.<n>GRAIL starts from fully specified 3D configurations in which object geometry, camera parameters, metric scale, environment depth, and a robot-proportioned character are known before video generation and reused during reconstruction.<n>We retarget the recovered motions to a humanoid robot and train complementary task-general morphology trackers.<n>GRAIL produces over 20,000 sequences spanning pick-up, object manipulation, sitting
  arXiv  Detail & Related papers  (2026-06-03T17:57:45Z)
• DynaFLIP: Rethinking Robotics Perception via Tri-Modal-Dynamics Guided Representation [54.860960182207236]
  We introduce DynaFLIP, a multimodal pre-training framework that pushes motion understanding upstream into perception.<n>We construct image-language-3D flow triplets from heterogeneous human and robot videos, and use these triplets as training-time supervision to shape an image-only encoder.<n>Our results suggest that robot generalization improves when visual representations are trained to encode not just what is present, but how the world changes under action.
  arXiv  Detail & Related papers  (2026-05-28T17:59:53Z)
• Lyra: Generative 3D Scene Reconstruction via Video Diffusion Model Self-Distillation [87.91642226587294]
  Current learning-based 3D reconstruction methods rely on the availability of captured real-world multi-view data.<n>We propose a self-distillation framework that distills the implicit 3D knowledge in the video diffusion models into an explicit 3D Gaussian Splatting (3DGS) representation.<n>Our framework achieves state-of-the-art performance in static and dynamic 3D scene generation.
  arXiv  Detail & Related papers  (2025-09-23T17:58:01Z)
• QuaDreamer: Controllable Panoramic Video Generation for Quadruped Robots [35.458219603280995]
  QuaDreamer is the first panoramic data generation engine specifically designed for quadruped robots.<n>To facilitate high-quality panoramic video generation under jitter signal control, we propose a Scene-Object Controller (SOC)<n>The generated video sequences can serve as training data for the quadruped robot's panoramic visual perception model.
  arXiv  Detail & Related papers  (2025-08-04T15:18:01Z)
• Geometry-aware 4D Video Generation for Robot Manipulation [28.709339959536106]
  We propose a 4D video generation model that enforces multi-view 3D consistency of videos by supervising the model with cross-view pointmap alignment during training.<n>This geometric supervision enables the model to learn a shared 3D representation of the scene, allowing it to predict future video sequences from novel viewpoints.<n>Compared to existing baselines, our method produces more visually stable and spatially aligned predictions across multiple simulated and real-world robotic datasets.
  arXiv  Detail & Related papers  (2025-07-01T18:01:41Z)
• VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulation [53.63540587160549]
  VidBot is a framework enabling zero-shot robotic manipulation using learned 3D affordance from in-the-wild monocular RGB-only human videos.<n> VidBot paves the way for leveraging everyday human videos to make robot learning more scalable.
  arXiv  Detail & Related papers  (2025-03-10T10:04:58Z)
• BAA-NGP: Bundle-Adjusting Accelerated Neural Graphics Primitives [6.431806897364565]
  Implicit neural representations have become pivotal in robotic perception, enabling robots to comprehend 3D environments from 2D images. We propose a framework called bundle-adjusting accelerated neural graphics primitives (BAA-NGP) Results demonstrate 10 to 20 x speed improvement compared to other bundle-adjusting neural radiance field methods.
  arXiv  Detail & Related papers  (2023-06-07T05:36:45Z)
• COPILOT: Human-Environment Collision Prediction and Localization from Egocentric Videos [62.34712951567793]
  The ability to forecast human-environment collisions from egocentric observations is vital to enable collision avoidance in applications such as VR, AR, and wearable assistive robotics. We introduce the challenging problem of predicting collisions in diverse environments from multi-view egocentric videos captured from body-mounted cameras. We propose a transformer-based model called COPILOT to perform collision prediction and localization simultaneously.
  arXiv  Detail & Related papers  (2022-10-04T17:49:23Z)
• Neural Scene Representation for Locomotion on Structured Terrain [56.48607865960868]
  We propose a learning-based method to reconstruct the local terrain for a mobile robot traversing urban environments. Using a stream of depth measurements from the onboard cameras and the robot's trajectory, the estimates the topography in the robot's vicinity. We propose a 3D reconstruction model that faithfully reconstructs the scene, despite the noisy measurements and large amounts of missing data coming from the blind spots of the camera arrangement.
  arXiv  Detail & Related papers  (2022-06-16T10:45:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.