Unfolding a blurred image
- URL: http://arxiv.org/abs/2201.12010v1
- Date: Fri, 28 Jan 2022 09:39:55 GMT
- Title: Unfolding a blurred image
- Authors: Kuldeep Purohit, Anshul Shah, A. N. Rajagopalan
- Abstract summary: We learn motion representation from sharp videos in an unsupervised manner.
We then train a convolutional recurrent video autoencoder network that performs a surrogate task of video reconstruction.
It is employed for guided training of a motion encoder for blurred images.
This network extracts embedded motion information from the blurred image to generate a sharp video in conjunction with the trained recurrent video decoder.
- Score: 36.519356428362286
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We present a solution for the goal of extracting a video from a single motion
blurred image to sequentially reconstruct the clear views of a scene as beheld
by the camera during the time of exposure. We first learn motion representation
from sharp videos in an unsupervised manner through training of a convolutional
recurrent video autoencoder network that performs a surrogate task of video
reconstruction. Once trained, it is employed for guided training of a motion
encoder for blurred images. This network extracts embedded motion information
from the blurred image to generate a sharp video in conjunction with the
trained recurrent video decoder. As an intermediate step, we also design an
efficient architecture that enables real-time single image deblurring and
outperforms competing methods across all factors: accuracy, speed, and
compactness. Experiments on real scenes and standard datasets demonstrate the
superiority of our framework over the state-of-the-art and its ability to
generate a plausible sequence of temporally consistent sharp frames.
Related papers
- Neuromorphic Synergy for Video Binarization [54.195375576583864]
Bimodal objects serve as a visual form to embed information that can be easily recognized by vision systems.
Neuromorphic cameras offer new capabilities for alleviating motion blur, but it is non-trivial to first de-blur and then binarize the images in a real-time manner.
We propose an event-based binary reconstruction method that leverages the prior knowledge of the bimodal target's properties to perform inference independently in both event space and image space.
We also develop an efficient integration method to propagate this binary image to high frame rate binary video.
arXiv Detail & Related papers (2024-02-20T01:43:51Z) - Concatenated Masked Autoencoders as Spatial-Temporal Learner [6.475592804311682]
We introduce the Concatenated Masked Autoencoders (CatMAE) as a spatial-temporal learner for self-supervised video representation learning.
We propose a new data augmentation strategy, Video-Reverse (ViRe), which uses reversed video frames as the model's reconstruction targets.
arXiv Detail & Related papers (2023-11-02T03:08:26Z) - Aggregating Long-term Sharp Features via Hybrid Transformers for Video
Deblurring [76.54162653678871]
We propose a video deblurring method that leverages both neighboring frames and present sharp frames using hybrid Transformers for feature aggregation.
Our proposed method outperforms state-of-the-art video deblurring methods as well as event-driven video deblurring methods in terms of quantitative metrics and visual quality.
arXiv Detail & Related papers (2023-09-13T16:12:11Z) - Frozen CLIP Models are Efficient Video Learners [86.73871814176795]
Video recognition has been dominated by the end-to-end learning paradigm.
Recent advances in Contrastive Vision-Language Pre-training pave the way for a new route for visual recognition tasks.
We present Efficient Video Learning -- an efficient framework for directly training high-quality video recognition models.
arXiv Detail & Related papers (2022-08-06T17:38:25Z) - Video Reconstruction from a Single Motion Blurred Image using Learned
Dynamic Phase Coding [34.76550131783525]
We propose a hybrid optical-digital method for video reconstruction using a single motion-blurred image.
We use a learned dynamic phase-coding in the lens aperture during the image acquisition to encode the motion trajectories.
The proposed computational camera generates a sharp frame burst of the scene at various frame rates from a single coded motion-blurred image.
arXiv Detail & Related papers (2021-12-28T02:06:44Z) - Restoration of Video Frames from a Single Blurred Image with Motion
Understanding [69.90724075337194]
We propose a novel framework to generate clean video frames from a single motion-red image.
We formulate video restoration from a single blurred image as an inverse problem by setting clean image sequence and their respective motion as latent factors.
Our framework is based on anblur-decoder structure with spatial transformer network modules.
arXiv Detail & Related papers (2021-04-19T08:32:57Z) - Motion-blurred Video Interpolation and Extrapolation [72.3254384191509]
We present a novel framework for deblurring, interpolating and extrapolating sharp frames from a motion-blurred video in an end-to-end manner.
To ensure temporal coherence across predicted frames and address potential temporal ambiguity, we propose a simple, yet effective flow-based rule.
arXiv Detail & Related papers (2021-03-04T12:18:25Z) - Video Reconstruction by Spatio-Temporal Fusion of Blurred-Coded Image
Pair [16.295479896947853]
Recovering video from a single motion-blurred image is a very ill-posed problem.
Traditional coded exposure framework is better-posed but it only samples a fraction of the space-time volume.
We propose to use the complementary information present in the fully-exposed image along with the coded exposure image to recover a high fidelity video.
arXiv Detail & Related papers (2020-10-20T06:08:42Z) - Task-agnostic Temporally Consistent Facial Video Editing [84.62351915301795]
We propose a task-agnostic, temporally consistent facial video editing framework.
Based on a 3D reconstruction model, our framework is designed to handle several editing tasks in a more unified and disentangled manner.
Compared with the state-of-the-art facial image editing methods, our framework generates video portraits that are more photo-realistic and temporally smooth.
arXiv Detail & Related papers (2020-07-03T02:49:20Z) - Learning to Deblur and Generate High Frame Rate Video with an Event
Camera [0.0]
Event cameras do not suffer from motion blur when recording high-speed scenes.
We formulate the deblurring task on traditional cameras directed by events to be a residual learning one.
We propose corresponding network architectures for effective learning of deblurring and high frame rate video generation tasks.
arXiv Detail & Related papers (2020-03-02T13:02:05Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.