Tele360: Real-Time Feed-Forward Human Reconstruction from Sparse Unposed Cameras

• URL: http://arxiv.org/abs/2609.15032v1
• Date: Mon, 14 Sep 2026 04:50:52 GMT
• Title: Tele360: Real-Time Feed-Forward Human Reconstruction from Sparse Unposed Cameras
• Abstract summary: We present Tele360, the first real-time feed-forward system for dynamic human reconstruction and live free-viewpoint visualization.<n>Our system jointly estimates camera poses and reconstructs a dynamic 3D Gaussian representation for each time instance.<n>Experiments show that Tele360 achieves state-of-the-art visual quality on studio benchmarks while supporting real-time 2K input-to-rendering at over 25 FPS on a single consumer GPU.
• Score: 43.56045236607455
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Live free-viewpoint visualization of real humans is critical for immersive communication and interactive digital experiences. Existing methods either rely on computationally expensive optimization or require calibrated cameras and low-resolution inputs, making real-time high-resolution deployment impractical. In this work, we present Tele360, the first real-time feed-forward system for dynamic human reconstruction and live free-viewpoint visualization from sparse, unposed RGB streams. Our system jointly estimates camera poses and reconstructs a dynamic 3D Gaussian representation for each time instance in a single forward pass. To achieve this, we start by designing a lightweight sparsity-aware multi-view transformer backbone that tokenizes foreground human regions while preserving global context through a shared scene token. We then employ a fully transformer-based Gaussian decoder to mitigate convolution-induced over-smoothing while keeping decoding sparse and efficient. In addition, we introduce a hybrid feature pyramid that injects multi-scale appearance cues into geometry prediction. We further introduce a lightweight differentiable Levenberg-Marquardt camera refinement layer to enhance multi-view consistency and geometric alignment. Moreover, to stabilize learning under sparse, unposed inputs, we transfer multi-view geometry priors from a large visual-geometry foundation model via teacher-student distillation. Finally, the predicted Gaussian maps are streamed with video codecs to remote devices for interactive free-viewpoint rendering. Extensive experiments show that Tele360 achieves state-of-the-art visual quality on studio benchmarks while supporting real-time 2K input-to-rendering at over 25 FPS on a single consumer GPU. Additional captured sequences illustrate its performance across varied subjects, clothing, and motions under our multi-camera setup.

Related papers

• FF-ProCams: Feed-Forward Gaussian Splatting for Projector-Camera System [17.828959053537798]
  Projector-camera (ProCams) systems achieve active scene perception and controllable appearance manipulation via structured illumination.<n>Existing inverse-rendering methods for ProCams deliver high-fidelity results but rely on time-consuming per-scene optimization.<n>We propose FF-ProCams, a Feed-Forward 3D Gaussian inverse-rendering framework for ProCams.
  arXiv  Detail & Related papers  (2026-07-20T10:42:16Z)
• ReViV: Reconstructing the Viewer and the View in 4D from Monocular Egocentric Video [51.951128822418774]
  ReViV is a framework for holistic egocentric 4D reconstruction.<n>Powered by a Masked Generative Egocentric Transformer, ReViV operates within a single feed-forward architecture.
  arXiv  Detail & Related papers  (2026-07-20T10:22:53Z)
• Geometrically Consistent Multi-View Scene Generation from Freehand Sketches [58.98194920417429]
  Freehand sketches are the most geometrically impoverished input one could offer a multi-view generator.<n>We address three compounding challenges; absent training data, the need for geometric reasoning from distorted 2D input, and cross-view consistency.<n>Our framework synthesizes all views in a single denoising process without requiring reference images, iterative refinement, or per-scene optimisation.
  arXiv  Detail & Related papers  (2026-04-15T18:00:45Z)
• RobotPan: A 360$^\circ$ Surround-View Robotic Vision System for Embodied Perception [47.76543396190029]
  We introduce a surround-view robotic vision system that combines six cameras with LiDAR to provide full 360$circ$ visual coverage.<n>We also present textscRobotPan, a feed-forward framework that predicts emphmetric-scaled and emphcompact 3D Gaussians from calibrated sparse-view inputs.<n>Experiments show that textscRobotPan achieves competitive quality against prior feed-forward reconstruction and view-synthesis methods.
  arXiv  Detail & Related papers  (2026-04-15T04:58:23Z)
• ViewMorpher3D: A 3D-aware Diffusion Framework for Multi-Camera Novel View Synthesis in Autonomous Driving [20.935790354765604]
  We introduce ViewMorpher3D, a multi-view image enhancement framework based on image diffusion models.<n>Unlike single-view approaches, ViewMorpher3D jointly processes a set of rendered views conditioned on camera poses, 3D geometric priors, and temporally adjacent or spatially overlapping reference views.<n>Our framework accommodates variable numbers of cameras and flexible reference/target view configurations, making it adaptable to diverse sensor setups.
  arXiv  Detail & Related papers  (2026-01-12T13:44:14Z)
• Mon3tr: Monocular 3D Telepresence with Pre-built Gaussian Avatars as Amortization [16.68162021163563]
  Mon3tr is a novel Monocular 3D telepresence framework that integrates 3D Gaussian splatting (3DGS) based parametric human modeling.<n>A single monocular RGB camera is used to capture body motions and facial expressions in real time to drive the 3DGS-based parametric human model.<n>Our method achieves a PSNR of > 28 dB for novel poses, an end-to-end latency of 80 ms, and > 1000x bandwidth reduction compared to point-cloud streaming.
  arXiv  Detail & Related papers  (2026-01-12T13:17:41Z)
• Visual Odometry with Transformers [68.453547770334]
  We introduce Visual odometry Transformer (VoT), which processes sequences of monocular frames by extracting features.<n>Unlike prior methods, VoT directly predicts camera motion without estimating dense geometry and relies solely on camera poses for supervision.<n>VoT scales effectively with larger datasets, benefits substantially from stronger pre-trained backbones, generalizes across diverse camera motions and calibration settings, and outperforms traditional methods while running more than 3 times faster.
  arXiv  Detail & Related papers  (2025-10-02T17:00:14Z)
• Learning Dynamic View Synthesis With Few RGBD Cameras [60.36357774688289]
  We propose to utilize RGBD cameras to synthesize free-viewpoint videos of dynamic indoor scenes. We generate point clouds from RGBD frames and then render them into free-viewpoint videos via a neural feature. We introduce a simple Regional Depth-Inpainting module that adaptively inpaints missing depth values to render complete novel views.
  arXiv  Detail & Related papers  (2022-04-22T03:17:35Z)
• Real-time dense 3D Reconstruction from monocular video data captured by low-cost UAVs [0.3867363075280543]
  Real-time 3D reconstruction enables fast dense mapping of the environment which benefits numerous applications, such as navigation or live evaluation of an emergency. In contrast to most real-time capable approaches, our approach does not need an explicit depth sensor. By exploiting the self-motion of the unmanned aerial vehicle (UAV) flying with oblique view around buildings, we estimate both camera trajectory and depth for selected images with enough novel content.
  arXiv  Detail & Related papers  (2021-04-21T13:12:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.