Spatio-Temporal Difference Guided Motion Deblurring with the Complementary Vision Sensor

• URL: http://arxiv.org/abs/2604.10554v1
• Date: Sun, 12 Apr 2026 09:54:14 GMT
• Title: Spatio-Temporal Difference Guided Motion Deblurring with the Complementary Vision Sensor
• Abstract summary: Motion blur arises when rapid scene changes occur during the exposure period, collapsing rich intra-exposure motion into a single RGB frame.<n>We propose Spatio-Temporal Difference Guided Deblur Net (STGDNet) to restore structure and color details lost in blurry RGB inputs.<n>Our method outperforms current RGB or event-based approaches in both synthetic CVS dataset and real-world evaluations.
• Score: 12.418360718687026
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Motion blur arises when rapid scene changes occur during the exposure period, collapsing rich intra-exposure motion into a single RGB frame. Without explicit structural or temporal cues, RGB-only deblurring is highly ill-posed and often fails under extreme motion. Inspired by the human visual system, brain-inspired vision sensors introduce temporally dense information to alleviate this problem. However, event cameras still suffer from event rate saturation under rapid motion, while the event modality entangles edge features and motion cues, which limits their effectiveness. As a recent breakthrough, the complementary vision sensor (CVS), Tianmouc, captures synchronized RGB frames together with high-frame-rate, multi-bit spatial difference (SD, encoding structural edges) and temporal difference (TD, encoding motion cues) data within a single RGB exposure, offering a promising solution for RGB deblurring under extreme dynamic scenes. To fully leverage these complementary modalities, we propose Spatio-Temporal Difference Guided Deblur Net (STGDNet), which adopts a recurrent multi-branch architecture that iteratively encodes and fuses SD and TD sequences to restore structure and color details lost in blurry RGB inputs. Our method outperforms current RGB or event-based approaches in both synthetic CVS dataset and real-world evaluations. Moreover, STGDNet exhibits strong generalization capability across over 100 extreme real-world scenarios. Project page: https://tmcDeblur.github.io/

Related papers

• Cyclops: LiDAR as a Camera That Dreams in Color [32.25721672776894]
  We propose a framework that translates sparse Non-Repetitive Scanning LiDAR (NRS-LiDAR) intensity into RGB video, enabling camera-free inference for all-day perception tasks.<n>Our approach first converts sparse LiDAR intensity projections into dense representations via a frozen pre-trained densification module.<n>The dense intensity latent is then transported toward the target RGB distribution through Latent Bridge Matching.
  arXiv  Detail & Related papers  (2026-08-17T08:37:09Z)
• ENCORE: Event-Assisted Complementary Motion Refinement for Learned Video Compression [16.49973800168176]
  Event-Assisted Complementary Motion Refinement framework for learned video compression.<n> ENCORE achieves up to 20.80% PSNR-RGB and 22.14% MS-SSIM-RGB BD-rate savings.<n> experiments on BS-ERGB, HQ-EVFI, and CED demonstrate consistent gains across datasets and GOP lengths.
  arXiv  Detail & Related papers  (2026-07-30T11:10:13Z)
• RainDancer: RGB-Event Video Deraining with Rain-Oriented Spiking Dynamics [93.19893734404417]
  Video deraining aims to recover clean visual content from rainy videos for reliable perception under adverse weather.<n>Existing methods mainly rely on RGB sequences and temporal redundancy, but RGB-only restoration remains ambiguous in dynamic rainy scenes.<n>We propose RainDancer, a progressive RGB-Event video deraining framework based on a decompose-before-interact paradigm.
  arXiv  Detail & Related papers  (2026-07-15T13:08:21Z)
• UniRED: Unified RGB-D Video Frame Interpolation with Event Guidance [19.80142530142947]
  High frame-rate RGB-D videos are crucial for a variety of downstream tasks, including motion analysis, dynamic scene understanding, and 3D reconstruction.<n>Existing video methods have achieved strong performance on RGB data, but they are not readily applicable to RGB-D scenarios.<n>We propose a unified multimodal framework for RGB-D video that jointly exploits RGB appearance, depth geometry, and event-based temporal cues.
  arXiv  Detail & Related papers  (2026-06-23T08:03:58Z)
• E-RGB-D: Real-Time Event-Based Perception with Structured Light [4.894362825271711]
  Event-based cameras (ECs) have emerged as bio-inspired sensors that report pixel brightness changes asynchronously.<n>We present a novel approach that integrates a Digital Light Processing (DLP) projector, forming Active Structured Light (ASL) for RGB-D sensing.
  arXiv  Detail & Related papers  (2025-12-20T17:08:11Z)
• Human Activity Recognition using RGB-Event based Sensors: A Multi-modal Heat Conduction Model and A Benchmark Dataset [65.76480665062363]
  Human Activity Recognition primarily relied on traditional RGB cameras to achieve high-performance activity recognition.<n>Challenges in real-world scenarios, such as insufficient lighting and rapid movements, inevitably degrade the performance of RGB cameras.<n>In this work, we rethink human activity recognition by combining the RGB and event cameras.
  arXiv  Detail & Related papers  (2025-04-08T09:14:24Z)
• Dynamic EventNeRF: Reconstructing General Dynamic Scenes from Multi-view RGB and Event Streams [69.65147723239153]
  Volumetric reconstruction of dynamic scenes is an important problem in computer vision.<n>It is especially challenging in poor lighting and with fast motion.<n>We propose the first method totemporally reconstruct a scene from sparse multi-view event streams and sparse RGB frames.
  arXiv  Detail & Related papers  (2024-12-09T18:56:18Z)
• Complementing Event Streams and RGB Frames for Hand Mesh Reconstruction [51.87279764576998]
  We propose EvRGBHand -- the first approach for 3D hand mesh reconstruction with an event camera and an RGB camera compensating for each other. EvRGBHand can tackle overexposure and motion blur issues in RGB-based HMR and foreground scarcity and background overflow issues in event-based HMR.
  arXiv  Detail & Related papers  (2024-03-12T06:04:50Z)
• Implicit Event-RGBD Neural SLAM [54.74363487009845]
  Implicit neural SLAM has achieved remarkable progress recently. Existing methods face significant challenges in non-ideal scenarios. We propose EN-SLAM, the first event-RGBD implicit neural SLAM framework.
  arXiv  Detail & Related papers  (2023-11-18T08:48:58Z)
• Revisiting Event-based Video Frame Interpolation [49.27404719898305]
  Dynamic vision sensors or event cameras provide rich complementary information for video frame. estimating optical flow from events is arguably more difficult than from RGB information. We propose a divide-and-conquer strategy in which event-based intermediate frame synthesis happens incrementally in multiple simplified stages.
  arXiv  Detail & Related papers  (2023-07-24T06:51:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.