Generalizing Event-Based Motion Deblurring in Real-World Scenarios

• URL: http://arxiv.org/abs/2308.05932v1
• Date: Fri, 11 Aug 2023 04:27:29 GMT
• Title: Generalizing Event-Based Motion Deblurring in Real-World Scenarios
• Authors: Xiang Zhang, Lei Yu, Wen Yang, Jianzhuang Liu, Gui-Song Xia
• Abstract summary: Event-based motion deblurring has shown promising results by exploiting low-latency events. We propose a scale-aware network that allows flexible input spatial scales and enables learning from different temporal scales of motion blur. A two-stage self-supervised learning scheme is then developed to fit real-world data distribution.
• Score: 62.995994797897424
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Event-based motion deblurring has shown promising results by exploiting low-latency events. However, current approaches are limited in their practical usage, as they assume the same spatial resolution of inputs and specific blurriness distributions. This work addresses these limitations and aims to generalize the performance of event-based deblurring in real-world scenarios. We propose a scale-aware network that allows flexible input spatial scales and enables learning from different temporal scales of motion blur. A two-stage self-supervised learning scheme is then developed to fit real-world data distribution. By utilizing the relativity of blurriness, our approach efficiently ensures the restored brightness and structure of latent images and further generalizes deblurring performance to handle varying spatial and temporal scales of motion blur in a self-distillation manner. Our method is extensively evaluated, demonstrating remarkable performance, and we also introduce a real-world dataset consisting of multi-scale blurry frames and events to facilitate research in event-based deblurring.

Related papers

• Rethinking Efficient and Effective Point-based Networks for Event Camera Classification and Regression: EventMamba [11.400397931501338]
  Event cameras efficiently detect changes in ambient light with low latency and high dynamic range while consuming minimal power. Most current approach to processing event data often involves converting it into frame-based representations. Point Cloud is a popular representation for 3D processing and is better suited to match the sparse and asynchronous nature of the event camera. We propose EventMamba, an efficient and effective Point Cloud framework that achieves competitive results even compared to the state-of-the-art (SOTA) frame-based method.
  arXiv  Detail & Related papers  (2024-05-09T21:47:46Z)
• Implicit Event-RGBD Neural SLAM [54.74363487009845]
  Implicit neural SLAM has achieved remarkable progress recently. Existing methods face significant challenges in non-ideal scenarios. We propose EN-SLAM, the first event-RGBD implicit neural SLAM framework.
  arXiv  Detail & Related papers  (2023-11-18T08:48:58Z)
• CrossZoom: Simultaneously Motion Deblurring and Event Super-Resolving [38.96663258582471]
  CrossZoom is a novel unified neural Network (CZ-Net) to jointly recover sharp latent sequences within the exposure period of a blurry input and the corresponding High-Resolution (HR) events. We present a multi-scale blur-event fusion architecture that leverages the scale-variant properties and effectively fuses cross-modality information to achieve cross-enhancement. We propose a new dataset containing HR sharp-blurry images and the corresponding HR-LR event streams to facilitate future research.
  arXiv  Detail & Related papers  (2023-09-29T03:27:53Z)
• Event-based Simultaneous Localization and Mapping: A Comprehensive Survey [52.73728442921428]
  Review of event-based vSLAM algorithms that exploit the benefits of asynchronous and irregular event streams for localization and mapping tasks. Paper categorizes event-based vSLAM methods into four main categories: feature-based, direct, motion-compensation, and deep learning methods.
  arXiv  Detail & Related papers  (2023-04-19T16:21:14Z)
• Adaptive Local-Component-aware Graph Convolutional Network for One-shot Skeleton-based Action Recognition [54.23513799338309]
  We present an Adaptive Local-Component-aware Graph Convolutional Network for skeleton-based action recognition. Our method provides a stronger representation than the global embedding and helps our model reach state-of-the-art.
  arXiv  Detail & Related papers  (2022-09-21T02:33:07Z)
• Motion Deblurring with Real Events [50.441934496692376]
  We propose an end-to-end learning framework for event-based motion deblurring in a self-supervised manner. Real-world events are exploited to alleviate the performance degradation caused by data inconsistency.
  arXiv  Detail & Related papers  (2021-09-28T13:11:44Z)
• Unsupervised Feature Learning for Event Data: Direct vs Inverse Problem Formulation [53.850686395708905]
  Event-based cameras record an asynchronous stream of per-pixel brightness changes. In this paper, we focus on single-layer architectures for representation learning from event data. We show improvements of up to 9 % in the recognition accuracy compared to the state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-09-23T10:40:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.