Towards Rolling Shutter Correction and Deblurring in Dynamic Scenes
- URL: http://arxiv.org/abs/2104.01601v1
- Date: Sun, 4 Apr 2021 12:36:48 GMT
- Title: Towards Rolling Shutter Correction and Deblurring in Dynamic Scenes
- Authors: Zhihang Zhong, Yinqiang Zheng and Imari Sato
- Abstract summary: Joint rolling shutter correction and deblurring (RSCD) techniques are critical for CMOS cameras.
We contribute the first dataset, BS-RSCD, which includes both ego-motion and object-motion in dynamic scenes.
Real distorted and blurry videos with corresponding ground truth are recorded simultaneously via a beam-splitter-based acquisition system.
- Score: 45.663838355778786
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Joint rolling shutter correction and deblurring (RSCD) techniques are
critical for the prevalent CMOS cameras. However, current approaches are still
based on conventional energy optimization and are developed for static scenes.
To enable learning-based approaches to address real-world RSCD problem, we
contribute the first dataset, BS-RSCD, which includes both ego-motion and
object-motion in dynamic scenes. Real distorted and blurry videos with
corresponding ground truth are recorded simultaneously via a
beam-splitter-based acquisition system.
Since direct application of existing individual rolling shutter correction
(RSC) or global shutter deblurring (GSD) methods on RSCD leads to undesirable
results due to inherent flaws in the network architecture, we further present
the first learning-based model (JCD) for RSCD. The key idea is that we adopt
bi-directional warping streams for displacement compensation, while also
preserving the non-warped deblurring stream for details restoration. The
experimental results demonstrate that JCD achieves state-of-the-art performance
on the realistic RSCD dataset (BS-RSCD) and the synthetic RSC dataset
(Fastec-RS). The dataset and code are available at
https://github.com/zzh-tech/RSCD.
Related papers
- SelfDRSC++: Self-Supervised Learning for Dual Reversed Rolling Shutter Correction [72.05587640928879]
We propose an enhanced Self-supervised learning framework for Dual reversed RS distortion Correction (SelfDRSC++)
We introduce a lightweight DRSC network that incorporates a bidirectional correlation matching block to refine the joint optimization of optical flows and corrected RS features.
To effectively train the DRSC network, we propose a self-supervised learning strategy that ensures cycle consistency between input and reconstructed dual reversed RS images.
arXiv Detail & Related papers (2024-08-21T08:17:22Z) - Single Image Rolling Shutter Removal with Diffusion Models [46.57721145372241]
We present RS-Diffusion, the first Diffusion Models-based method for single-frame Rolling Shutter (RS) correction.
In this work, we present an image-to-motion'' framework via diffusion techniques, with a designed patch-attention module.
In addition, we present the RS-Real dataset, comprised of captured RS frames alongside their corresponding Global Shutter (GS) ground-truth pairs.
arXiv Detail & Related papers (2024-07-03T08:25:02Z) - Self-supervised Learning to Bring Dual Reversed Rolling Shutter Images
Alive [56.70381414277253]
We propose a Self-supervised learning framework for Dual RS distortions Correction (SelfDRSC)
A DRSC network can be learned to generate a high framerate GS video only based on dual RS images with reversed distortions.
On real-world RS cases, our SelfDRSC can produce framerate high-of-the-art videos with finer correction textures and better temporary consistency.
arXiv Detail & Related papers (2023-05-31T13:55:00Z) - Self-Supervised Scene Dynamic Recovery from Rolling Shutter Images and
Events [63.984927609545856]
Event-based Inter/intra-frame Compensator (E-IC) is proposed to predict the per-pixel dynamic between arbitrary time intervals.
We show that the proposed method achieves state-of-the-art and shows remarkable performance for event-based RS2GS inversion in real-world scenarios.
arXiv Detail & Related papers (2023-04-14T05:30:02Z) - Learning Adaptive Warping for Real-World Rolling Shutter Correction [52.45689075940234]
This paper proposes the first real-world rolling shutter (RS) correction dataset, BS-RSC, and a corresponding model to correct the RS frames in a distorted video.
Mobile devices in the consumer market with CMOS-based sensors for video capture often result in rolling shutter effects when relative movements occur during the video acquisition process.
arXiv Detail & Related papers (2022-04-29T05:13:50Z) - Bringing Rolling Shutter Images Alive with Dual Reversed Distortion [75.78003680510193]
Rolling shutter (RS) distortion can be interpreted as the result of picking a row of pixels from instant global shutter (GS) frames over time.
We develop a novel end-to-end model, IFED, to generate dual optical flow sequence through iterative learning of the velocity field during the RS time.
arXiv Detail & Related papers (2022-03-12T14:57:49Z) - Deep network for rolling shutter rectification [25.170821013431958]
We propose an end-to-end deep neural network for the challenging task of single image RS rectification.
Our network consists of a motion block, a trajectory module, a row block, an RS rectification module and an RS regeneration module.
Experiments on synthetic and real datasets reveal that our network outperforms prior art both qualitatively and quantitatively.
arXiv Detail & Related papers (2021-12-12T06:40:34Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.