WildRayZer: Self-supervised Large View Synthesis in Dynamic Environments

• URL: http://arxiv.org/abs/2601.10716v1
• Date: Thu, 15 Jan 2026 18:59:58 GMT
• Title: WildRayZer: Self-supervised Large View Synthesis in Dynamic Environments
• Abstract summary: We present WildRayZer, a framework for novel view synthesis (NVS) in dynamic environments where both the camera and objects move.<n>Experiments show that WildRayZer consistently outperforms optimization-based and feed-forward baselines in both transient-region removal and full-frame NVS quality.
• Score: 12.449699948290215
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present WildRayZer, a self-supervised framework for novel view synthesis (NVS) in dynamic environments where both the camera and objects move. Dynamic content breaks the multi-view consistency that static NVS models rely on, leading to ghosting, hallucinated geometry, and unstable pose estimation. WildRayZer addresses this by performing an analysis-by-synthesis test: a camera-only static renderer explains rigid structure, and its residuals reveal transient regions. From these residuals, we construct pseudo motion masks, distill a motion estimator, and use it to mask input tokens and gate loss gradients so supervision focuses on cross-view background completion. To enable large-scale training and evaluation, we curate Dynamic RealEstate10K (D-RE10K), a real-world dataset of 15K casually captured dynamic sequences, and D-RE10K-iPhone, a paired transient and clean benchmark for sparse-view transient-aware NVS. Experiments show that WildRayZer consistently outperforms optimization-based and feed-forward baselines in both transient-region removal and full-frame NVS quality with a single feed-forward pass.

Related papers

• Dynamic-Robust Photometric-Semantic Reconstruction for Open-Vocabulary 3D Scene Understanding [63.599342750566485]
  SPAR is a novel joint semantic-geometric encoding architecture that explicitly isolates transient dynamic noise prior to latent space aggregation.<n>Our end-to-end approach achieves exceptional novel view synthesis quality, yielding a PSNR of 22.15 dB and 23.33 dB given only 3 and 4 input views respectively.
  arXiv  Detail & Related papers  (2026-08-29T09:58:24Z)
• RayDer: Scalable Self-Supervised Novel View Synthesis from Real-World Video [24.21520175195074]
  RayDer is a feed-forward transformer that consolidates camera estimation, scene reconstruction, and rendering into a single backbone.<n>A minimal dynamic state, treated as a nuisance factor, absorbs time-varying content and enables stable training on real-world video.<n>RayDer exhibits clean power-law scaling with data and compute, and outperforms static-scene data mixtures.
  arXiv  Detail & Related papers  (2026-05-29T16:50:27Z)
• WildPose: A Unified Framework for Robust Pose Estimation in the Wild [13.59516321308291]
  We present WildPose, a unified monocular pose framework that is robust in dynamic environments.<n>Our key insight is to connect two powerful paradigms in modern 3D vision: the rich perceptual estimation of feedforward models and the end-to-end optimization of differentiable bundle adjustment (BA)
  arXiv  Detail & Related papers  (2026-05-12T21:39:44Z)
• Accurate Open-Loop Control of a Soft Continuum Robot Through Visually Learned Latent Representations [4.857795247230421]
  This work addresses open-loop control of a soft continuum robot (SCR) from video-learned latent dynamics.<n>ABCD-based models consistently reduce image-space tracking error.<n>Several architecture choices and training settings contribute to the open-loop control performance.
  arXiv  Detail & Related papers  (2026-03-20T05:43:26Z)
• From Frames to Sequences: Temporally Consistent Human-Centric Dense Prediction [22.291273919939957]
  We develop a scalable synthetic data pipeline that generates human frames and motion-aligned sequences with pixel-accurate depth, normals, and masks.<n>We train a unified ViT-based dense predictor that injects an explicit geometric human prior via CSE embeddings.<n>Our two-stage training strategy, combining static pretraining with dynamic sequence supervision, enables the model first to acquire robust spatial representations and then refine temporal consistency across motion-aligned sequences.
  arXiv  Detail & Related papers  (2026-02-02T05:28:58Z)
• DVS-PedX: Synthetic-and-Real Event-Based Pedestrian Dataset [0.0]
  Event cameras like Dynamic Vision Sensors (DVS) report micro-timed brightness changes instead of full frames.<n>DVS-PedX is a neuromorphic dataset designed for pedestrian detection and crossing-intention analysis in normal and adverse weather conditions.
  arXiv  Detail & Related papers  (2025-09-04T11:30:29Z)
• UP-SLAM: Adaptively Structured Gaussian SLAM with Uncertainty Prediction in Dynamic Environments [1.4117999181375773]
  UP-SLAM is a real-time RGB-D SLAM system for dynamic environments.<n>It decouples tracking and mapping through a parallelized framework.<n>Experiments suggest it outperforms state-of-the-art methods in both localization accuracy and rendering quality.
  arXiv  Detail & Related papers  (2025-05-28T13:23:16Z)
• DATAP-SfM: Dynamic-Aware Tracking Any Point for Robust Structure from Motion in the Wild [85.03973683867797]
  This paper proposes a concise, elegant, and robust pipeline to estimate smooth camera trajectories and obtain dense point clouds for casual videos in the wild. We show that the proposed method achieves state-of-the-art performance in terms of camera pose estimation even in complex dynamic challenge scenes.
  arXiv  Detail & Related papers  (2024-11-20T13:01:16Z)
• D-NPC: Dynamic Neural Point Clouds for Non-Rigid View Synthesis from Monocular Video [53.83936023443193]
  This paper contributes to the field by introducing a new synthesis method for dynamic novel view from monocular video, such as smartphone captures.<n>Our approach represents the as a $textitdynamic neural point cloud$, an implicit time-conditioned point cloud that encodes local geometry and appearance in separate hash-encoded neural feature grids.
  arXiv  Detail & Related papers  (2024-06-14T14:35:44Z)
• Multi-Level Neural Scene Graphs for Dynamic Urban Environments [64.26401304233843]
  We present a novel, decomposable radiance field approach for dynamic urban environments. We propose a multi-level neural scene graph representation that scales to thousands of images from dozens of sequences with hundreds of fast-moving objects.
  arXiv  Detail & Related papers  (2024-03-29T21:52:01Z)
• DynaMoN: Motion-Aware Fast and Robust Camera Localization for Dynamic Neural Radiance Fields [71.94156412354054]
  We propose Dynamic Motion-Aware Fast and Robust Camera Localization for Dynamic Neural Radiance Fields (DynaMoN)<n>DynaMoN handles dynamic content for initial camera pose estimation and statics-focused ray sampling for fast and accurate novel-view synthesis.<n>We extensively evaluate our approach on two real-world dynamic datasets, the TUM RGB-D dataset and the BONN RGB-D Dynamic dataset.
  arXiv  Detail & Related papers  (2023-09-16T08:46:59Z)
• Dynamic 3D Gaussians: Tracking by Persistent Dynamic View Synthesis [58.5779956899918]
  We present a method that simultaneously addresses the tasks of dynamic scene novel-view synthesis and six degree-of-freedom (6-DOF) tracking of all dense scene elements. We follow an analysis-by-synthesis framework, inspired by recent work that models scenes as a collection of 3D Gaussians. We demonstrate a large number of downstream applications enabled by our representation, including first-person view synthesis, dynamic compositional scene synthesis, and 4D video editing.
  arXiv  Detail & Related papers  (2023-08-18T17:59:21Z)
• SUDS: Scalable Urban Dynamic Scenes [46.965165390077146]
  We extend neural radiance fields (NeRFs) to dynamic large-scale urban scenes. We factorize the scene into three separate hash table data structures to efficiently encode static, dynamic, and far-field radiance fields. Our reconstructions can be scaled to tens of thousands of objects across 1.2 million frames from 1700 videos spanning geospatial footprints of hundreds of kilometers.
  arXiv  Detail & Related papers  (2023-03-25T18:55:09Z)
• Learning Dynamic View Synthesis With Few RGBD Cameras [60.36357774688289]
  We propose to utilize RGBD cameras to synthesize free-viewpoint videos of dynamic indoor scenes. We generate point clouds from RGBD frames and then render them into free-viewpoint videos via a neural feature. We introduce a simple Regional Depth-Inpainting module that adaptively inpaints missing depth values to render complete novel views.
  arXiv  Detail & Related papers  (2022-04-22T03:17:35Z)
• Editable Free-viewpoint Video Using a Layered Neural Representation [35.44420164057911]
  We propose the first approach for editable free-viewpoint video generation for large-scale dynamic scenes using only sparse 16 cameras. The core of our approach is a new layered neural representation, where each dynamic entity including the environment itself is formulated into a space-time coherent neural layered radiance representation called ST-NeRF. Experiments demonstrate the effectiveness of our approach to achieve high-quality, photo-realistic, and editable free-viewpoint video generation for dynamic scenes.
  arXiv  Detail & Related papers  (2021-04-30T06:50:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.