Event-Based Dense Reconstruction Pipeline
- URL: http://arxiv.org/abs/2203.12270v1
- Date: Wed, 23 Mar 2022 08:37:04 GMT
- Title: Event-Based Dense Reconstruction Pipeline
- Authors: Kun Xiao, Guohui Wang, Yi Chen, Jinghong Nan, Yongfeng Xie
- Abstract summary: Event cameras are a new type of sensors that are different from traditional cameras.
Deep learning is used to reconstruct intensity images from events.
structure from motion (SfM) is used to estimate camera intrinsic, extrinsic and sparse point cloud.
- Score: 5.341354397748495
- License: http://creativecommons.org/publicdomain/zero/1.0/
- Abstract: Event cameras are a new type of sensors that are different from traditional
cameras. Each pixel is triggered asynchronously by event. The trigger event is
the change of the brightness irradiated on the pixel. If the increment or
decrement of brightness is higher than a certain threshold, an event is output.
Compared with traditional cameras, event cameras have the advantages of high
dynamic range and no motion blur. Since events are caused by the apparent
motion of intensity edges, the majority of 3D reconstructed maps consist only
of scene edges, i.e., semi-dense maps, which is not enough for some
applications. In this paper, we propose a pipeline to realize event-based dense
reconstruction. First, deep learning is used to reconstruct intensity images
from events. And then, structure from motion (SfM) is used to estimate camera
intrinsic, extrinsic and sparse point cloud. Finally, multi-view stereo (MVS)
is used to complete dense reconstruction.
Related papers
- Complementing Event Streams and RGB Frames for Hand Mesh Reconstruction [51.87279764576998]
We propose EvRGBHand -- the first approach for 3D hand mesh reconstruction with an event camera and an RGB camera compensating for each other.
EvRGBHand can tackle overexposure and motion blur issues in RGB-based HMR and foreground scarcity and background overflow issues in event-based HMR.
arXiv Detail & Related papers (2024-03-12T06:04:50Z) - Temporal-Mapping Photography for Event Cameras [5.838762448259289]
Event cameras capture brightness changes as a continuous stream of events'' rather than traditional intensity frames.
We realize events to dense intensity image conversion using a stationary event camera in static scenes.
arXiv Detail & Related papers (2024-03-11T05:29:46Z) - Robust e-NeRF: NeRF from Sparse & Noisy Events under Non-Uniform Motion [67.15935067326662]
Event cameras offer low power, low latency, high temporal resolution and high dynamic range.
NeRF is seen as the leading candidate for efficient and effective scene representation.
We propose Robust e-NeRF, a novel method to directly and robustly reconstruct NeRFs from moving event cameras.
arXiv Detail & Related papers (2023-09-15T17:52:08Z) - Dense Voxel 3D Reconstruction Using a Monocular Event Camera [5.599072208069752]
Event cameras offer many advantages over conventional frame-based cameras.
Their application in 3D reconstruction for VR applications is underexplored.
We propose a novel approach for solving dense 3D reconstruction using only a single event camera.
arXiv Detail & Related papers (2023-09-01T10:46:57Z) - Event-based Camera Tracker by $\nabla$t NeRF [11.572930535988325]
We show that we can recover the camera pose by minimizing the error between sparse events and the temporal gradient of the scene represented as a neural radiance field (NeRF)
We propose an event-based camera pose tracking framework called TeGRA which realizes the pose update by using the sparse event's observation.
arXiv Detail & Related papers (2023-04-07T16:03:21Z) - Multi-Event-Camera Depth Estimation and Outlier Rejection by Refocused
Events Fusion [14.15744053080529]
Event cameras are bio-inspired sensors that offer advantages over traditional cameras.
We tackle the problem of event-based stereo 3D reconstruction for SLAM.
We develop fusion theory and apply it to design multi-camera 3D reconstruction algorithms.
arXiv Detail & Related papers (2022-07-21T14:19:39Z) - A Preliminary Research on Space Situational Awareness Based on Event
Cameras [8.27218838055049]
Event camera is a new type of sensor that is different from traditional cameras.
The trigger event is the change of the brightness irradiated on the pixel.
Compared with traditional cameras, event cameras have the advantages of high temporal resolution, low latency, high dynamic range, low bandwidth and low power consumption.
arXiv Detail & Related papers (2022-03-24T14:36:18Z) - ESL: Event-based Structured Light [62.77144631509817]
Event cameras are bio-inspired sensors providing significant advantages over standard cameras.
We propose a novel structured-light system using an event camera to tackle the problem of accurate and high-speed depth sensing.
arXiv Detail & Related papers (2021-11-30T15:47:39Z) - Combining Events and Frames using Recurrent Asynchronous Multimodal
Networks for Monocular Depth Prediction [51.072733683919246]
We introduce Recurrent Asynchronous Multimodal (RAM) networks to handle asynchronous and irregular data from multiple sensors.
Inspired by traditional RNNs, RAM networks maintain a hidden state that is updated asynchronously and can be queried at any time to generate a prediction.
We show an improvement over state-of-the-art methods by up to 30% in terms of mean depth absolute error.
arXiv Detail & Related papers (2021-02-18T13:24:35Z) - EventHands: Real-Time Neural 3D Hand Reconstruction from an Event Stream [80.15360180192175]
3D hand pose estimation from monocular videos is a long-standing and challenging problem.
We address it for the first time using a single event camera, i.e., an asynchronous vision sensor reacting on brightness changes.
Our approach has characteristics previously not demonstrated with a single RGB or depth camera.
arXiv Detail & Related papers (2020-12-11T16:45:34Z) - EventSR: From Asynchronous Events to Image Reconstruction, Restoration,
and Super-Resolution via End-to-End Adversarial Learning [75.17497166510083]
Event cameras sense intensity changes and have many advantages over conventional cameras.
Some methods have been proposed to reconstruct intensity images from event streams.
The outputs are still in low resolution (LR), noisy, and unrealistic.
We propose a novel end-to-end pipeline that reconstructs LR images from event streams, enhances the image qualities and upsamples the enhanced images, called EventSR.
arXiv Detail & Related papers (2020-03-17T10:58:10Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.