Dynamic Scene Reconstruction: Recent Advance in Real-time Rendering and Streaming

• URL: http://arxiv.org/abs/2503.08166v1
• Date: Tue, 11 Mar 2025 08:29:41 GMT
• Title: Dynamic Scene Reconstruction: Recent Advance in Real-time Rendering and Streaming
• Authors: Jiaxuan Zhu, Hao Tang,
• Abstract summary: Representing and rendering dynamic scenes from 2D images is a fundamental yet challenging problem in computer vision and graphics.<n>This survey provides a comprehensive review of the evolution and advancements in dynamic scene representation and rendering.<n>We systematically summarize existing approaches, categorize them according to their core principles, compile relevant datasets, compare the performance of various methods on these benchmarks, and explore the challenges and future research directions in this rapidly evolving field.
• Score: 7.250878248686215
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Representing and rendering dynamic scenes from 2D images is a fundamental yet challenging problem in computer vision and graphics. This survey provides a comprehensive review of the evolution and advancements in dynamic scene representation and rendering, with a particular emphasis on recent progress in Neural Radiance Fields based and 3D Gaussian Splatting based reconstruction methods. We systematically summarize existing approaches, categorize them according to their core principles, compile relevant datasets, compare the performance of various methods on these benchmarks, and explore the challenges and future research directions in this rapidly evolving field. In total, we review over 170 relevant papers, offering a broad perspective on the state of the art in this domain.

Related papers

• Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Survey [154.50661618628433]
  3D reconstruction and view synthesis are foundational problems in computer vision, graphics, and immersive technologies such as augmented reality (AR), virtual reality (VR), and digital twins.<n>Recent advances in feed-forward approaches, driven by deep learning, have revolutionized this field by enabling fast and generalizable 3D reconstruction and view synthesis.
  arXiv  Detail & Related papers  (2025-07-19T06:13:25Z)
• Advances in Radiance Field for Dynamic Scene: From Neural Field to Gaussian Field [85.12359852781216]
  This survey presents a systematic analysis of over 200 papers focused on dynamic scene representation using radiance field.<n>We organize diverse methodological approaches under a unified representational framework, concluding with a critical examination of persistent challenges and promising research directions.
  arXiv  Detail & Related papers  (2025-05-15T07:51:08Z)
• A Survey of 3D Reconstruction with Event Cameras [16.103940503726022]
  Event cameras produce sparse yet temporally dense data streams, enabling robust and accurate 3D reconstruction.<n>These capabilities offer substantial promise for transformative applications across various fields, including autonomous driving, robotics, aerial navigation, and immersive virtual reality.<n>This survey aims to serve as an essential reference and provides a clear and motivating roadmap toward advancing the state of the art in event-driven 3D reconstruction.
  arXiv  Detail & Related papers  (2025-05-13T11:04:04Z)
• IDArb: Intrinsic Decomposition for Arbitrary Number of Input Views and Illuminations [64.07859467542664]
  Capturing geometric and material information from images remains a fundamental challenge in computer vision and graphics.<n>Traditional optimization-based methods often require hours of computational time to reconstruct geometry, material properties, and environmental lighting from dense multi-view inputs.<n>We introduce IDArb, a diffusion-based model designed to perform intrinsic decomposition on an arbitrary number of images under varying illuminations.
  arXiv  Detail & Related papers  (2024-12-16T18:52:56Z)
• Feed-Forward Bullet-Time Reconstruction of Dynamic Scenes from Monocular Videos [101.48581851337703]
  We present BTimer, the first motion-aware feed-forward model for real-time reconstruction and novel view synthesis of dynamic scenes. Our approach reconstructs the full scene in a 3D Gaussian Splatting representation at a given target ('bullet') timestamp by aggregating information from all the context frames. Given a casual monocular dynamic video, BTimer reconstructs a bullet-time scene within 150ms while reaching state-of-the-art performance on both static and dynamic scene datasets.
  arXiv  Detail & Related papers  (2024-12-04T18:15:06Z)
• GausSurf: Geometry-Guided 3D Gaussian Splatting for Surface Reconstruction [79.42244344704154]
  GausSurf employs geometry guidance from multi-view consistency in texture-rich areas and normal priors in texture-less areas of a scene.<n>Our method surpasses state-of-the-art methods in terms of reconstruction quality and computation time.
  arXiv  Detail & Related papers  (2024-11-29T03:54:54Z)
• ReMatching Dynamic Reconstruction Flow [55.272357926111454]
  This work introduces the ReMatching framework, designed to improve reconstruction quality by incorporating deformation priors into dynamic reconstruction models.<n>Our approach advocates for velocity-field based priors, for which we suggest a matching procedure that can seamlessly supplement existing dynamic reconstruction pipelines.<n>Our evaluations on popular benchmarks involving both synthetic and real-world dynamic scenes demonstrate that augmenting current state-of-the-art methods with our approach leads to a clear improvement in reconstruction accuracy.
  arXiv  Detail & Related papers  (2024-11-01T16:09:33Z)
• A Comprehensive Taxonomy and Analysis of Talking Head Synthesis: Techniques for Portrait Generation, Driving Mechanisms, and Editing [8.171572460041823]
  Talking head synthesis is an advanced method for generating portrait videos from a still image driven by specific content. This survey systematically reviews the technology, categorizing it into three pivotal domains: portrait generation, driven mechanisms, and editing techniques.
  arXiv  Detail & Related papers  (2024-06-15T08:14:59Z)
• Gaussian Splatting: 3D Reconstruction and Novel View Synthesis, a Review [0.08823202672546056]
  This review paper focuses on state-of-the-art techniques for 3D reconstruction, including the generation of novel, unseen views. An overview of recent developments in the Gaussian Splatting method is provided, covering input types, model structures, output representations, and training strategies.
  arXiv  Detail & Related papers  (2024-05-06T12:32:38Z)
• Scalable Scene Modeling from Perspective Imaging: Physics-based Appearance and Geometry Inference [3.2229099973277076]
  dissertation presents a fraction of contributions that advances 3D scene modeling to its state of the art. In contrast to the prevailing deep learning methods, as a core contribution, this thesis aims to develop algorithms that follow first principles.
  arXiv  Detail & Related papers  (2024-04-01T17:09:40Z)
• Recent Trends in 3D Reconstruction of General Non-Rigid Scenes [104.07781871008186]
  Reconstructing models of the real world, including 3D geometry, appearance, and motion of real scenes, is essential for computer graphics and computer vision. It enables the synthesizing of photorealistic novel views, useful for the movie industry and AR/VR applications. This state-of-the-art report (STAR) offers the reader a comprehensive summary of state-of-the-art techniques with monocular and multi-view inputs.
  arXiv  Detail & Related papers  (2024-03-22T09:46:11Z)
• Eye Motion Matters for 3D Face Reconstruction [13.633246294557765]
  We introduce an Eye Landmark Adjustment Module, complemented by a Local Dynamic Loss, to capture the dynamic features of the eyes area. Our module allows for flexible adjustment of landmarks, resulting in accurate recreation of various eye states.
  arXiv  Detail & Related papers  (2024-01-18T01:47:55Z)
• A Survey on 3D Gaussian Splatting [51.96747208581275]
  3D Gaussian splatting (GS) has emerged as a transformative technique in explicit radiance field and computer graphics.<n>We provide the first systematic overview of the recent developments and critical contributions in the domain of 3D GS.<n>By enabling unprecedented rendering speed, 3D GS opens up a plethora of applications, ranging from virtual reality to interactive media and beyond.
  arXiv  Detail & Related papers  (2024-01-08T13:42:59Z)
• Single-shot Tomography of Discrete Dynamic Objects [1.1407697960152927]
  We present a novel method for the reconstruction of high-resolution temporal images in dynamic tomographic imaging. The implications of this research extend to improved visualization and analysis of dynamic processes in tomographic imaging.
  arXiv  Detail & Related papers  (2023-11-09T10:52:02Z)
• DynaVol: Unsupervised Learning for Dynamic Scenes through Object-Centric Voxelization [67.85434518679382]
  We present DynaVol, a 3D scene generative model that unifies geometric structures and object-centric learning. The key idea is to perform object-centric voxelization to capture the 3D nature of the scene. voxel features evolve over time through a canonical-space deformation function, forming the basis for global representation learning.
  arXiv  Detail & Related papers  (2023-04-30T05:29:28Z)
• Neural 3D Reconstruction in the Wild [86.6264706256377]
  We introduce a new method that enables efficient and accurate surface reconstruction from Internet photo collections. We present a new benchmark and protocol for evaluating reconstruction performance on such in-the-wild scenes.
  arXiv  Detail & Related papers  (2022-05-25T17:59:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.