Dynamic View Synthesis as an Inverse Problem

• URL: http://arxiv.org/abs/2506.08004v1
• Date: Mon, 09 Jun 2025 17:59:47 GMT
• Title: Dynamic View Synthesis as an Inverse Problem
• Abstract summary: We address dynamic view synthesis from monocular videos as an inverse problem in a training-free setting.<n>We introduce a novel noise representation, termed K-order Recursive Noise Representation.<n>To synthesize newly visible regions from camera motion, we introduce Latent Modulation.
• Score: 3.7599363231894185
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work, we address dynamic view synthesis from monocular videos as an inverse problem in a training-free setting. By redesigning the noise initialization phase of a pre-trained video diffusion model, we enable high-fidelity dynamic view synthesis without any weight updates or auxiliary modules. We begin by identifying a fundamental obstacle to deterministic inversion arising from zero-terminal signal-to-noise ratio (SNR) schedules and resolve it by introducing a novel noise representation, termed K-order Recursive Noise Representation. We derive a closed form expression for this representation, enabling precise and efficient alignment between the VAE-encoded and the DDIM inverted latents. To synthesize newly visible regions resulting from camera motion, we introduce Stochastic Latent Modulation, which performs visibility aware sampling over the latent space to complete occluded regions. Comprehensive experiments demonstrate that dynamic view synthesis can be effectively performed through structured latent manipulation in the noise initialization phase.

Related papers

• Dynamic-Robust Photometric-Semantic Reconstruction for Open-Vocabulary 3D Scene Understanding [63.599342750566485]
  SPAR is a novel joint semantic-geometric encoding architecture that explicitly isolates transient dynamic noise prior to latent space aggregation.<n>Our end-to-end approach achieves exceptional novel view synthesis quality, yielding a PSNR of 22.15 dB and 23.33 dB given only 3 and 4 input views respectively.
  arXiv  Detail & Related papers  (2026-08-29T09:58:24Z)
• Retrieving and Refining Winning Noise Tickets for Diffusion-Based Motion Generation [32.824608968455294]
  Diffusion-based text-to-motion models synthesize realistic human motions but often exhibit semantic drift from the input text.<n>We propose WInning Noise Retrieval and Optimization (WINRO), a framework that improves text-motion alignment by selecting and refining such tickets before diffusion sampling.<n>WINRO consistently improves text-motion fidelity across different base models, MDM and MotionLCM, on HumanML3D without retraining.
  arXiv  Detail & Related papers  (2026-07-07T22:32:10Z)
• Implicit Neural Representations: A Signal Processing Perspective [54.57279006229212]
  Implicit neural representations (INRs) mark a fundamental shift in signal modeling, moving from discrete sampled data to continuous functional representations.<n>This article examines the evolution of INRs from a signal processing perspective, emphasizing spectral behavior, sampling theory, and multiscale representation.<n>We highlight the utility of INRs across a broad range of applications, including inverse problems in medical and radar imaging, compression, and 3D scene representation.
  arXiv  Detail & Related papers  (2026-04-16T14:12:06Z)
• MOSAIC-GS: Monocular Scene Reconstruction via Advanced Initialization for Complex Dynamic Environments [12.796165448365949]
  MOSAIC-GS is a novel, fully explicit, and computationally efficient approach for high-fidelity dynamic scene reconstruction from monocular videos.<n>We leverage multiple geometric cues, such as depth, optical flow, dynamic object segmentation, and point tracking.<n>We demonstrate that MOSAIC-GS achieves substantially faster optimization and rendering compared to existing methods.
  arXiv  Detail & Related papers  (2026-01-08T20:48:24Z)
• Towards Arbitrary Motion Completing via Hierarchical Continuous Representation [64.6525112550758]
  We propose a novel parametric activation-induced hierarchical implicit representation framework, called NAME, based on Implicit Representations (INRs)<n>Our method introduces a hierarchical temporal encoding mechanism that extracts features from motion sequences at multiple temporal scales, enabling effective capture of intricate temporal patterns.
  arXiv  Detail & Related papers  (2025-12-24T14:07:04Z)
• Interlaced dynamic XCT reconstruction with spatio-temporal implicit neural representations [0.0]
  We investigate the use of of-Implicit Neural Representations for dynamic X-ray computed tomography (XCT) reconstruction under interlaced acquisition schemes.<n>The proposed approach combines ADMM-based optimization with INCODE, a conditioning framework incorporating prior knowledge, to enable efficient convergence.<n>Across all settings, our model achieves strong performance robustness and outperforms Time-Inter Model-Based Iter Reconstruction (TIMBIR), a state-of-the-art model-based iterative method.
  arXiv  Detail & Related papers  (2025-10-09T01:33:58Z)
• DynaSplat: Dynamic-Static Gaussian Splatting with Hierarchical Motion Decomposition for Scene Reconstruction [9.391616497099422]
  We present DynaSplat, an approach that extends Gaussian Splatting to dynamic scenes.<n>We classify scene elements as static or dynamic through a novel fusion of deformation offset statistics and 2D motion flow consistency.<n>We then introduce a hierarchical motion modeling strategy that captures both coarse global transformations and fine-grained local movements.
  arXiv  Detail & Related papers  (2025-06-11T15:13:35Z)
• SHaDe: Compact and Consistent Dynamic 3D Reconstruction via Tri-Plane Deformation and Latent Diffusion [0.0]
  We present a novel framework for dynamic 3D scene reconstruction that integrates three key components.<n>An explicit tri-plane deformation field, a view-conditioned canonical field with spherical harmonics (SH) attention, and a temporally-aware latent diffusion prior.<n>Our method encodes 4D scenes using three 2D feature planes that evolve over time, enabling efficient compact representation.
  arXiv  Detail & Related papers  (2025-05-22T11:25:38Z)
• JointTuner: Appearance-Motion Adaptive Joint Training for Customized Video Generation [13.168628936598367]
  JointTuner is a novel adaptive joint training framework.<n>We develop Adaptive LoRA, which incorporates a context-aware gating mechanism.<n>Appearance-independent Temporal Loss is introduced to decouple motion patterns from intrinsic appearance.
  arXiv  Detail & Related papers  (2025-03-31T11:04:07Z)
• Event-boosted Deformable 3D Gaussians for Dynamic Scene Reconstruction [50.873820265165975]
  We introduce the first approach combining event cameras, which capture high-temporal-resolution, continuous motion data, with deformable 3D-GS for dynamic scene reconstruction.<n>We propose a GS-Threshold Joint Modeling strategy, creating a mutually reinforcing process that greatly improves both 3D reconstruction and threshold modeling.<n>We contribute the first event-inclusive 4D benchmark with synthetic and real-world dynamic scenes, on which our method achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2024-11-25T08:23:38Z)
• Gear-NeRF: Free-Viewpoint Rendering and Tracking with Motion-aware Spatio-Temporal Sampling [70.34875558830241]
  We present a way for learning a-temporal (4D) embedding, based on semantic semantic gears to allow for stratified modeling of dynamic regions of rendering the scene. At the same time, almost for free, our tracking approach enables free-viewpoint of interest - a functionality not yet achieved by existing NeRF-based methods.
  arXiv  Detail & Related papers  (2024-06-06T03:37:39Z)
• Enhancing Dynamic CT Image Reconstruction with Neural Fields and Optical Flow [0.0]
  We show the benefits of introducing explicit motion regularizers for dynamic inverse problems based on partial differential equations.<n>We also compare neural fields against a grid-based solver and show that the former outperforms the latter in terms of PSNR.
  arXiv  Detail & Related papers  (2024-06-03T13:07:29Z)
• NVS-Solver: Video Diffusion Model as Zero-Shot Novel View Synthesizer [48.57740681957145]
  We propose a new novel view synthesis (NVS) paradigm that operates textitwithout the need for training.<n>NVS-r adaptively modulates the diffusion sampling process with the given views to enable the creation of remarkable visual experiences.
  arXiv  Detail & Related papers  (2024-05-24T08:56:19Z)
• Diffusion Priors for Dynamic View Synthesis from Monocular Videos [59.42406064983643]
  Dynamic novel view synthesis aims to capture the temporal evolution of visual content within videos. We first finetune a pretrained RGB-D diffusion model on the video frames using a customization technique. We distill the knowledge from the finetuned model to a 4D representations encompassing both dynamic and static Neural Radiance Fields.
  arXiv  Detail & Related papers  (2024-01-10T23:26:41Z)
• IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis [90.03590032170169]
  We present intrinsic neural radiance fields, dubbed IntrinsicNeRF, which introduce intrinsic decomposition into the NeRF-based neural rendering method. Our experiments and editing samples on both object-specific/room-scale scenes and synthetic/real-word data demonstrate that we can obtain consistent intrinsic decomposition results.
  arXiv  Detail & Related papers  (2022-10-02T22:45:11Z)
• Dynamic View Synthesis from Dynamic Monocular Video [69.80425724448344]
  We present an algorithm for generating views at arbitrary viewpoints and any input time step given a monocular video of a dynamic scene. We show extensive quantitative and qualitative results of dynamic view synthesis from casually captured videos.
  arXiv  Detail & Related papers  (2021-05-13T17:59:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.