E-3DGS: Gaussian Splatting with Exposure and Motion Events

• URL: http://arxiv.org/abs/2410.16995v2
• Date: Fri, 11 Apr 2025 02:45:54 GMT
• Title: E-3DGS: Gaussian Splatting with Exposure and Motion Events
• Authors: Xiaoting Yin, Hao Shi, Yuhan Bao, Zhenshan Bing, Yiyi Liao, Kailun Yang, Kaiwei Wang,
• Abstract summary: E-3DGS sets a new benchmark for event-based 3D reconstruction with robust performance in challenging conditions.<n>We introduce EME-3D, a real-world 3D dataset with exposure events, motion events, camera calibration parameters, and sparse point clouds.
• Score: 29.042018288378447
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Achieving 3D reconstruction from images captured under optimal conditions has been extensively studied in the vision and imaging fields. However, in real-world scenarios, challenges such as motion blur and insufficient illumination often limit the performance of standard frame-based cameras in delivering high-quality images. To address these limitations, we incorporate a transmittance adjustment device at the hardware level, enabling event cameras to capture both motion and exposure events for diverse 3D reconstruction scenarios. Motion events (triggered by camera or object movement) are collected in fast-motion scenarios when the device is inactive, while exposure events (generated through controlled camera exposure) are captured during slower motion to reconstruct grayscale images for high-quality training and optimization of event-based 3D Gaussian Splatting (3DGS). Our framework supports three modes: High-Quality Reconstruction using exposure events, Fast Reconstruction relying on motion events, and Balanced Hybrid optimizing with initial exposure events followed by high-speed motion events. On the EventNeRF dataset, we demonstrate that exposure events significantly improve fine detail reconstruction compared to motion events and outperform frame-based cameras under challenging conditions such as low illumination and overexposure. Furthermore, we introduce EME-3D, a real-world 3D dataset with exposure events, motion events, camera calibration parameters, and sparse point clouds. Our method achieves faster and higher-quality reconstruction than event-based NeRF and is more cost-effective than methods combining event and RGB data. E-3DGS sets a new benchmark for event-based 3D reconstruction with robust performance in challenging conditions and lower hardware demands. The source code and dataset will be available at https://github.com/MasterHow/E-3DGS.

Related papers

• EBAD-Gaussian: Event-driven Bundle Adjusted Deblur Gaussian Splatting [21.46091843175779]
  Event-driven Bundle Adjusted Deblur Gaussian Splatting (EBAD-Gaussian) EBAD-Gaussian reconstructs sharp 3D Gaussians from event streams and severely blurred images. Experiments on synthetic and real-world datasets show that EBAD-Gaussian can achieve high-quality 3D scene reconstruction.
  arXiv  Detail & Related papers  (2025-04-14T09:17:00Z)
• Low-Light Image Enhancement using Event-Based Illumination Estimation [83.81648559951684]
  Low-light image enhancement (LLIE) aims to improve the visibility of images captured in poorly lit environments. This paper opens a new avenue from the perspective of estimating the illumination using ''temporal-mapping'' events. We construct a beam-splitter setup and collect EvLowLight dataset that includes images, temporal-mapping events, and motion events.
  arXiv  Detail & Related papers  (2025-04-13T00:01:33Z)
• EventEgo3D++: 3D Human Motion Capture from a Head-Mounted Event Camera [64.58147600753382]
  EventEgo3D++ is a monocular event camera with a fisheye lens for 3D human motion capture. Event cameras excel in high-speed scenarios and varying illumination due to their high temporal resolution. Our method supports real-time 3D pose updates at a rate of 140Hz.
  arXiv  Detail & Related papers  (2025-02-11T18:57:05Z)
• AE-NeRF: Augmenting Event-Based Neural Radiance Fields for Non-ideal Conditions and Larger Scene [31.142207770861457]
  We propose AE-NeRF to address the challenges of learning event-based NeRF from non-ideal conditions. Our method achieves a new state-of-the-art in event-based 3D reconstruction.
  arXiv  Detail & Related papers  (2025-01-06T07:00:22Z)
• EventSplat: 3D Gaussian Splatting from Moving Event Cameras for Real-time Rendering [7.392798832833857]
  Event cameras offer exceptional temporal resolution and a high dynamic range. We introduce a method for using event camera data in novel view synthesis via Gaussian Splatting.
  arXiv  Detail & Related papers  (2024-12-10T08:23:58Z)
• EF-3DGS: Event-Aided Free-Trajectory 3D Gaussian Splatting [76.02450110026747]
  Event cameras, inspired by biological vision, record pixel-wise intensity changes asynchronously with high temporal resolution. We propose Event-Aided Free-Trajectory 3DGS, which seamlessly integrates the advantages of event cameras into 3DGS. We evaluate our method on the public Tanks and Temples benchmark and a newly collected real-world dataset, RealEv-DAVIS.
  arXiv  Detail & Related papers  (2024-10-20T13:44:24Z)
• IncEventGS: Pose-Free Gaussian Splatting from a Single Event Camera [7.515256982860307]
  IncEventGS is an incremental 3D Gaussian splatting reconstruction algorithm with a single event camera. We exploit the tracking and mapping paradigm of conventional SLAM pipelines for IncEventGS.
  arXiv  Detail & Related papers  (2024-10-10T16:54:23Z)
• Deblur e-NeRF: NeRF from Motion-Blurred Events under High-speed or Low-light Conditions [56.84882059011291]
  We propose Deblur e-NeRF, a novel method to reconstruct blur-minimal NeRFs from motion-red events. We also introduce a novel threshold-normalized total variation loss to improve the regularization of large textureless patches.
  arXiv  Detail & Related papers  (2024-09-26T15:57:20Z)
• Elite-EvGS: Learning Event-based 3D Gaussian Splatting by Distilling Event-to-Video Priors [8.93657924734248]
  Event cameras are bio-inspired sensors that output asynchronous and sparse event streams, instead of fixed frames. We propose a novel event-based 3DGS framework, named Elite-EvGS. Our key idea is to distill the prior knowledge from the off-the-shelf event-to-video (E2V) models to effectively reconstruct 3D scenes from events.
  arXiv  Detail & Related papers  (2024-09-20T10:47:52Z)
• Deblurring Neural Radiance Fields with Event-driven Bundle Adjustment [23.15130387716121]
  We propose Bundle Adjustment for Deblurring Neural Radiance Fields (EBAD-NeRF) to jointly optimize the learnable poses and NeRF parameters. EBAD-NeRF can obtain accurate camera trajectory during the exposure time and learn a sharper 3D representations compared to prior works.
  arXiv  Detail & Related papers  (2024-06-20T14:33:51Z)
• Event3DGS: Event-Based 3D Gaussian Splatting for High-Speed Robot Egomotion [54.197343533492486]
  Event3DGS can reconstruct high-fidelity 3D structure and appearance under high-speed egomotion. Experiments on multiple synthetic and real-world datasets demonstrate the superiority of Event3DGS compared with existing event-based dense 3D scene reconstruction frameworks. Our framework also allows one to incorporate a few motion-blurred frame-based measurements into the reconstruction process to further improve appearance fidelity without loss of structural accuracy.
  arXiv  Detail & Related papers  (2024-06-05T06:06:03Z)
• EvaGaussians: Event Stream Assisted Gaussian Splatting from Blurry Images [39.584967370302735]
  3D Gaussian Splatting (3D-GS) has demonstrated exceptional capabilities in 3D scene reconstruction and novel view synthesis. We introduce Event Stream Assisted Gaussian Splatting (EvaGaussians), a novel approach that integrates event streams captured by an event camera to assist in reconstructing high-quality 3D-GS from blurry images.
  arXiv  Detail & Related papers  (2024-05-29T04:59:27Z)
• EventEgo3D: 3D Human Motion Capture from Egocentric Event Streams [59.77837807004765]
  This paper introduces a new problem, i.e., 3D human motion capture from an egocentric monocular event camera with a fisheye lens. Event streams have high temporal resolution and provide reliable cues for 3D human motion capture under high-speed human motions and rapidly changing illumination. Our EE3D demonstrates robustness and superior 3D accuracy compared to existing solutions while supporting real-time 3D pose update rates of 140Hz.
  arXiv  Detail & Related papers  (2024-04-12T17:59:47Z)
• Complementing Event Streams and RGB Frames for Hand Mesh Reconstruction [51.87279764576998]
  We propose EvRGBHand -- the first approach for 3D hand mesh reconstruction with an event camera and an RGB camera compensating for each other. EvRGBHand can tackle overexposure and motion blur issues in RGB-based HMR and foreground scarcity and background overflow issues in event-based HMR.
  arXiv  Detail & Related papers  (2024-03-12T06:04:50Z)
• EventAid: Benchmarking Event-aided Image/Video Enhancement Algorithms with Real-captured Hybrid Dataset [55.12137324648253]
  Event cameras are emerging imaging technology that offers advantages over conventional frame-based imaging sensors in dynamic range and sensing speed. This paper focuses on five event-aided image and video enhancement tasks.
  arXiv  Detail & Related papers  (2023-12-13T15:42:04Z)
• EventNeRF: Neural Radiance Fields from a Single Colour Event Camera [81.19234142730326]
  This paper proposes the first approach for 3D-consistent, dense and novel view synthesis using just a single colour event stream as input. At its core is a neural radiance field trained entirely in a self-supervised manner from events while preserving the original resolution of the colour event channels. We evaluate our method qualitatively and numerically on several challenging synthetic and real scenes and show that it produces significantly denser and more visually appealing renderings.
  arXiv  Detail & Related papers  (2022-06-23T17:59:53Z)
• ESL: Event-based Structured Light [62.77144631509817]
  Event cameras are bio-inspired sensors providing significant advantages over standard cameras. We propose a novel structured-light system using an event camera to tackle the problem of accurate and high-speed depth sensing.
  arXiv  Detail & Related papers  (2021-11-30T15:47:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.