Event Enhanced High-Quality Image Recovery

• URL: http://arxiv.org/abs/2007.08336v1
• Date: Thu, 16 Jul 2020 13:51:15 GMT
• Title: Event Enhanced High-Quality Image Recovery
• Authors: Bishan Wang, Jingwei He, Lei Yu, Gui-Song Xia, Wen Yang
• Abstract summary: We propose an explainable network, an event-enhanced sparse learning network (eSL-Net) to recover the high-quality images from event cameras. After training with a synthetic dataset, the proposed eSL-Net can largely improve the performance of the state-of-the-art by 7-12 dB.
• Score: 34.46486617222021
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With extremely high temporal resolution, event cameras have a large potential for robotics and computer vision. However, their asynchronous imaging mechanism often aggravates the measurement sensitivity to noises and brings a physical burden to increase the image spatial resolution. To recover high-quality intensity images, one should address both denoising and super-resolution problems for event cameras. Since events depict brightness changes, with the enhanced degeneration model by the events, the clear and sharp high-resolution latent images can be recovered from the noisy, blurry and low-resolution intensity observations. Exploiting the framework of sparse learning, the events and the low-resolution intensity observations can be jointly considered. Based on this, we propose an explainable network, an event-enhanced sparse learning network (eSL-Net), to recover the high-quality images from event cameras. After training with a synthetic dataset, the proposed eSL-Net can largely improve the performance of the state-of-the-art by 7-12 dB. Furthermore, without additional training process, the proposed eSL-Net can be easily extended to generate continuous frames with frame-rate as high as the events.

Related papers

• Recovering Continuous Scene Dynamics from A Single Blurry Image with Events [58.7185835546638]
  An Implicit Video Function (IVF) is learned to represent a single motion blurred image with concurrent events. A dual attention transformer is proposed to efficiently leverage merits from both modalities. The proposed network is trained only with the supervision of ground-truth images of limited referenced timestamps.
  arXiv  Detail & Related papers  (2023-04-05T18:44:17Z)
• Learning to Super-Resolve Blurry Images with Events [62.61911224564196]
  Super-Resolution from a single motion Blurred image (SRB) is a severely ill-posed problem due to the joint degradation of motion blurs and low spatial resolution. We employ events to alleviate the burden of SRB and propose an Event-enhanced SRB (E-SRB) algorithm. We show that the proposed eSL-Net++ outperforms state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2023-02-27T13:46:42Z)
• Learning to See Through with Events [37.19232535463858]
  This paper presents an Event-based SAI (E-SAI) method by relying on asynchronous events with extremely low latency and high dynamic range. The collected events are first refocused by a Re-focus-Net module to align in-focus events while scattering out off-focus ones. A hybrid network composed of spiking neural networks (SNNs) and convolutional neural networks (CNNs) is proposed to encode the foreground-temporal information from the refocused events and reconstruct a visual image of the occluded scenes.
  arXiv  Detail & Related papers  (2022-12-05T12:51:22Z)
• Ev-NeRF: Event Based Neural Radiance Field [8.78321125097048]
  Ev-NeRF is a Neural Radiance Field derived from event data. We show that Ev-NeRF achieves competitive performance for intensity image reconstruction under extreme noise conditions.
  arXiv  Detail & Related papers  (2022-06-24T18:27:30Z)
• Are High-Resolution Event Cameras Really Needed? [62.70541164894224]
  In low-illumination conditions and at high speeds, low-resolution cameras can outperform high-resolution ones, while requiring a significantly lower bandwidth. We provide both empirical and theoretical evidence for this claim, which indicates that high-resolution event cameras exhibit higher per-pixel event rates. In most cases, high-resolution event cameras show a lower task performance, compared to lower resolution sensors in these conditions.
  arXiv  Detail & Related papers  (2022-03-28T12:06:20Z)
• MEFNet: Multi-scale Event Fusion Network for Motion Deblurring [62.60878284671317]
  Traditional frame-based cameras inevitably suffer from motion blur due to long exposure times. As a kind of bio-inspired camera, the event camera records the intensity changes in an asynchronous way with high temporal resolution. In this paper, we rethink the event-based image deblurring problem and unfold it into an end-to-end two-stage image restoration network.
  arXiv  Detail & Related papers  (2021-11-30T23:18:35Z)
• Event-based Synthetic Aperture Imaging with a Hybrid Network [30.178111153441666]
  We propose a novel SAI system based on the event camera which can produce asynchronous events with extremely low latency and high dynamic range. To reconstruct the occluded targets, we propose a hybrid encoder-decoder network composed of spiking neural networks (SNNs) and convolutional neural networks (CNNs)
  arXiv  Detail & Related papers  (2021-03-03T12:56:55Z)
• Learning to Detect Objects with a 1 Megapixel Event Camera [14.949946376335305]
  Event cameras encode visual information with high temporal precision, low data-rate, and high-dynamic range. Due to the novelty of the field, the performance of event-based systems on many vision tasks is still lower compared to conventional frame-based solutions.
  arXiv  Detail & Related papers  (2020-09-28T16:03:59Z)
• EventSR: From Asynchronous Events to Image Reconstruction, Restoration, and Super-Resolution via End-to-End Adversarial Learning [75.17497166510083]
  Event cameras sense intensity changes and have many advantages over conventional cameras. Some methods have been proposed to reconstruct intensity images from event streams. The outputs are still in low resolution (LR), noisy, and unrealistic. We propose a novel end-to-end pipeline that reconstructs LR images from event streams, enhances the image qualities and upsamples the enhanced images, called EventSR.
  arXiv  Detail & Related papers  (2020-03-17T10:58:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.