Ev4DGS: Novel-view Rendering of Non-Rigid Objects from Monocular Event Streams

• URL: http://arxiv.org/abs/2510.11717v1
• Date: Mon, 13 Oct 2025 17:59:55 GMT
• Title: Ev4DGS: Novel-view Rendering of Non-Rigid Objects from Monocular Event Streams
• Authors: Takuya Nakabayashi, Navami Kairanda, Hideo Saito, Vladislav Golyanik,
• Abstract summary: Ev4DGS is the first approach for novel view rendering of non-rigidly deforming objects in the explicit observation space.<n>Our method regresses a deformable 3D Gaussian Splatting representation through 1) a loss relating the outputs of the estimated model with the 2D event observation space, and 2) a coarse 3D deformation model trained from binary masks generated from events.
• Score: 34.02850677061797
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Event cameras offer various advantages for novel view rendering compared to synchronously operating RGB cameras, and efficient event-based techniques supporting rigid scenes have been recently demonstrated in the literature. In the case of non-rigid objects, however, existing approaches additionally require sparse RGB inputs, which can be a substantial practical limitation; it remains unknown if similar models could be learned from event streams only. This paper sheds light on this challenging open question and introduces Ev4DGS, i.e., the first approach for novel view rendering of non-rigidly deforming objects in the explicit observation space (i.e., as RGB or greyscale images) from monocular event streams. Our method regresses a deformable 3D Gaussian Splatting representation through 1) a loss relating the outputs of the estimated model with the 2D event observation space, and 2) a coarse 3D deformation model trained from binary masks generated from events. We perform experimental comparisons on existing synthetic and newly recorded real datasets with non-rigid objects. The results demonstrate the validity of Ev4DGS and its superior performance compared to multiple naive baselines that can be applied in our setting. We will release our models and the datasets used in the evaluation for research purposes; see the project webpage: https://4dqv.mpi-inf.mpg.de/Ev4DGS/.

Related papers

• EventNeuS: 3D Mesh Reconstruction from a Single Event Camera [69.48085586670054]
  Event cameras offer an alternative to RGB cameras in many scenarios.<n>We present EventNeuS, a self-supervised neural model for learning 3D representations from monocular colour event streams.
  arXiv  Detail & Related papers  (2026-02-03T18:59:57Z)
• GS2E: Gaussian Splatting is an Effective Data Generator for Event Stream Generation [32.13436507983477]
  We introduce GS2E (Gaussian Splatting to Event), a large-scale synthetic event dataset for high-fidelity event vision tasks.<n>Results on event-based 3D reconstruction demonstrate GS2E's superior generalization capabilities and its practical value.
  arXiv  Detail & Related papers  (2025-05-21T09:15:42Z)
• Towards Human-Level 3D Relative Pose Estimation: Generalizable, Training-Free, with Single Reference [65.42565481489132]
  Humans can easily deduce the relative pose of a previously unseen object, without labeling or training, given only a single query-reference image pair.<n>We propose a novel 3D generalizable relative pose estimation method by elaborating 3D/2.5D shape perception with a 2.5D shape from RGB-D reference.<n>Our differentiable takes the 2.5D rotatable mesh textured by the RGB and the semantic maps (obtained by DINOv2 from the RGB input), then renders new RGB and semantic maps under a novel rotated view.
  arXiv  Detail & Related papers  (2024-06-26T16:01:10Z)
• PT43D: A Probabilistic Transformer for Generating 3D Shapes from Single Highly-Ambiguous RGB Images [26.900974153235456]
  We propose a transformer-based autoregressive model to generate 3D shapes from single RGB images. To handle realistic scenarios such as field-of-view truncation, we create simulated image-to-shape training pairs. We then adopt cross-attention to effectively identify the most relevant region of interest from the input image for shape generation.
  arXiv  Detail & Related papers  (2024-05-20T09:49:13Z)
• Zero123-6D: Zero-shot Novel View Synthesis for RGB Category-level 6D Pose Estimation [66.3814684757376]
  This work presents Zero123-6D, the first work to demonstrate the utility of Diffusion Model-based novel-view-synthesizers in enhancing RGB 6D pose estimation at category-level. The outlined method shows reduction in data requirements, removal of the necessity of depth information in zero-shot category-level 6D pose estimation task, and increased performance, quantitatively demonstrated through experiments on the CO3D dataset.
  arXiv  Detail & Related papers  (2024-03-21T10:38:18Z)
• EventNeRF: Neural Radiance Fields from a Single Colour Event Camera [81.19234142730326]
  This paper proposes the first approach for 3D-consistent, dense and novel view synthesis using just a single colour event stream as input. At its core is a neural radiance field trained entirely in a self-supervised manner from events while preserving the original resolution of the colour event channels. We evaluate our method qualitatively and numerically on several challenging synthetic and real scenes and show that it produces significantly denser and more visually appealing renderings.
  arXiv  Detail & Related papers  (2022-06-23T17:59:53Z)
• Learning Dynamic View Synthesis With Few RGBD Cameras [60.36357774688289]
  We propose to utilize RGBD cameras to synthesize free-viewpoint videos of dynamic indoor scenes. We generate point clouds from RGBD frames and then render them into free-viewpoint videos via a neural feature. We introduce a simple Regional Depth-Inpainting module that adaptively inpaints missing depth values to render complete novel views.
  arXiv  Detail & Related papers  (2022-04-22T03:17:35Z)
• RandomRooms: Unsupervised Pre-training from Synthetic Shapes and Randomized Layouts for 3D Object Detection [138.2892824662943]
  A promising solution is to make better use of the synthetic dataset, which consists of CAD object models, to boost the learning on real datasets. Recent work on 3D pre-training exhibits failure when transfer features learned on synthetic objects to other real-world applications. In this work, we put forward a new method called RandomRooms to accomplish this objective.
  arXiv  Detail & Related papers  (2021-08-17T17:56:12Z)
• Differentiable Event Stream Simulator for Non-Rigid 3D Tracking [82.56690776283428]
  Our differentiable simulator enables non-rigid 3D tracking of deformable objects from event streams. We show the effectiveness of our approach for various types of non-rigid objects and compare to existing methods for non-rigid 3D tracking.
  arXiv  Detail & Related papers  (2021-04-30T17:58:07Z)
• Category-Level 3D Non-Rigid Registration from Single-View RGB Images [28.874008960264202]
  We propose a novel approach to solve the 3D non-rigid registration problem from RGB images using CNNs. Our objective is to find a deformation field that warps a given 3D canonical model into a novel instance observed by a single-view RGB image.
  arXiv  Detail & Related papers  (2020-08-17T10:35:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.