Image Reconstruction from Events. Why learn it?

• URL: http://arxiv.org/abs/2112.06242v1
• Date: Sun, 12 Dec 2021 14:01:09 GMT
• Title: Image Reconstruction from Events. Why learn it?
• Authors: Zelin Zhang, Anthony Yezzi, Guillermo Gallego
• Abstract summary: We show how tackling the joint problem of motion estimation leads us to model event-based image reconstruction as a linear inverse problem. We propose classical and learning-based image priors can be used to solve the problem and remove artifacts from the reconstructed images.
• Score: 11.773972029187433
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Traditional cameras measure image intensity. Event cameras, by contrast, measure per-pixel temporal intensity changes asynchronously. Recovering intensity from events is a popular research topic since the reconstructed images inherit the high dynamic range (HDR) and high-speed properties of events; hence they can be used in many robotic vision applications and to generate slow-motion HDR videos. However, state-of-the-art methods tackle this problem by training an event-to-image recurrent neural network (RNN), which lacks explainability and is difficult to tune. In this work we show, for the first time, how tackling the joint problem of motion and intensity estimation leads us to model event-based image reconstruction as a linear inverse problem that can be solved without training an image reconstruction RNN. Instead, classical and learning-based image priors can be used to solve the problem and remove artifacts from the reconstructed images. The experiments show that the proposed approach generates images with visual quality on par with state-of-the-art methods despite only using data from a short time interval (i.e., without recurrent connections). Our method can also be used to improve the quality of images reconstructed by approaches that first estimate the image Laplacian; here our method can be interpreted as Poisson reconstruction guided by image priors.

Related papers

• NODER: Image Sequence Regression Based on Neural Ordinary Differential Equations [2.711538918087856]
  We propose an optimization-based new framework called NODER, which leverages neural ordinary differential equations to capture complex underlying dynamics. Our model needs only a couple of images in a sequence for prediction, which is practical, especially for clinical situations.
  arXiv  Detail & Related papers  (2024-07-18T07:50:46Z)
• MRI Field-transfer Reconstruction with Limited Data: Regularization by Neural Style Transfer [1.755209318470883]
  Regularization by denoising (RED) is a general pipeline which embeds a denoiser as a prior for image reconstruction. We propose a regularization by neural style transfer (RNST) method to further leverage the priors from the neural transfer and denoising engine.
  arXiv  Detail & Related papers  (2023-08-21T18:26:35Z)
• Single-Image HDR Reconstruction by Multi-Exposure Generation [8.656080193351581]
  We propose a weakly supervised learning method that inverts the physical image formation process for HDR reconstruction. Our neural network can invert the camera response to reconstruct pixel irradiance before synthesizing multiple exposures. Our experiments show that our proposed model can effectively reconstruct HDR images.
  arXiv  Detail & Related papers  (2022-10-28T05:12:56Z)
• Is Deep Image Prior in Need of a Good Education? [57.3399060347311]
  Deep image prior was introduced as an effective prior for image reconstruction. Despite its impressive reconstructive properties, the approach is slow when compared to learned or traditional reconstruction techniques. We develop a two-stage learning paradigm to address the computational challenge.
  arXiv  Detail & Related papers  (2021-11-23T15:08:26Z)
• Restormer: Efficient Transformer for High-Resolution Image Restoration [118.9617735769827]
  convolutional neural networks (CNNs) perform well at learning generalizable image priors from large-scale data. Transformers have shown significant performance gains on natural language and high-level vision tasks. Our model, named Restoration Transformer (Restormer), achieves state-of-the-art results on several image restoration tasks.
  arXiv  Detail & Related papers  (2021-11-18T18:59:10Z)
• How to cheat with metrics in single-image HDR reconstruction [19.361293438802264]
  Single-image high dynamic range (SI-The) reconstruction has recently emerged as a problem well-suited for deep learning methods. This paper highlights that such improvements in objective metrics do not necessarily translate to visually superior images. We show how such results are not representative of the perceived quality and that SI- reconstruction needs better evaluation protocols.
  arXiv  Detail & Related papers  (2021-08-19T14:29:15Z)
• Image Restoration by Deep Projected GSURE [115.57142046076164]
  Ill-posed inverse problems appear in many image processing applications, such as deblurring and super-resolution. We propose a new image restoration framework that is based on minimizing a loss function that includes a "projected-version" of the Generalized SteinUnbiased Risk Estimator (GSURE) and parameterization of the latent image by a CNN.
  arXiv  Detail & Related papers  (2021-02-04T08:52:46Z)
• Consistent Video Depth Estimation [57.712779457632024]
  We present an algorithm for reconstructing dense, geometrically consistent depth for all pixels in a monocular video. We leverage a conventional structure-from-motion reconstruction to establish geometric constraints on pixels in the video. Our algorithm is able to handle challenging hand-held captured input videos with a moderate degree of dynamic motion.
  arXiv  Detail & Related papers  (2020-04-30T17:59:26Z)
• Single-Image HDR Reconstruction by Learning to Reverse the Camera Pipeline [100.5353614588565]
  We propose to incorporate the domain knowledge of the LDR image formation pipeline into our model. We model the HDRto-LDR image formation pipeline as the (1) dynamic range clipping, (2) non-linear mapping from a camera response function, and (3) quantization. We demonstrate that the proposed method performs favorably against state-of-the-art single-image HDR reconstruction algorithms.
  arXiv  Detail & Related papers  (2020-04-02T17:59:04Z)
• Deep Blind Video Super-resolution [85.79696784460887]
  We propose a deep convolutional neural network (CNN) model to solve video SR by a blur kernel modeling approach. The proposed CNN model consists of motion blur estimation, motion estimation, and latent image restoration modules. We show that the proposed algorithm is able to generate clearer images with finer structural details.
  arXiv  Detail & Related papers  (2020-03-10T13:43:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.