Related papers: Video Deblurring by Fitting to Test Data

Video Deblurring by Fitting to Test Data

URL: http://arxiv.org/abs/2012.05228v2
Date: Sat, 6 Mar 2021 07:22:22 GMT
Title: Video Deblurring by Fitting to Test Data
Authors: Xuanchi Ren, Zian Qian, Qifeng Chen
Abstract summary: Motion blur in videos captured by autonomous vehicles and robots can degrade their perception capability. We present a novel approach to video deblurring by fitting a deep network to the test video. Our approach selects sharp frames from a video and then trains a convolutional neural network on these sharp frames.
Score: 39.41334067434719
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Abstract: Motion blur in videos captured by autonomous vehicles and robots can degrade their perception capability. In this work, we present a novel approach to video deblurring by fitting a deep network to the test video. Our key observation is that some frames in a video with motion blur are much sharper than others, and thus we can transfer the texture information in those sharp frames to blurry frames. Our approach heuristically selects sharp frames from a video and then trains a convolutional neural network on these sharp frames. The trained network often absorbs enough details in the scene to perform deblurring on all the video frames. As an internal learning method, our approach has no domain gap between training and test data, which is a problematic issue for existing video deblurring approaches. The conducted experiments on real-world video data show that our model can reconstruct clearer and sharper videos than state-of-the-art video deblurring approaches. Code and data are available at https://github.com/xrenaa/Deblur-by-Fitting.

Related papers

Aggregating Long-term Sharp Features via Hybrid Transformers for Video Deblurring [76.54162653678871]
We propose a video deblurring method that leverages both neighboring frames and present sharp frames using hybrid Transformers for feature aggregation. Our proposed method outperforms state-of-the-art video deblurring methods as well as event-driven video deblurring methods in terms of quantitative metrics and visual quality.
arXiv Detail & Related papers (2023-09-13T16:12:11Z)
Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generators [70.17041424896507]
Recent text-to-video generation approaches rely on computationally heavy training and require large-scale video datasets. We propose a new task of zero-shot text-to-video generation using existing text-to-image synthesis methods. Our method performs comparably or sometimes better than recent approaches, despite not being trained on additional video data.
arXiv Detail & Related papers (2023-03-23T17:01:59Z)
Low Light Video Enhancement by Learning on Static Videos with Cross-Frame Attention [10.119600046984088]
We develop a deep learning method for low light video enhancement by training a model on static videos. Existing methods operate frame by frame and do not exploit the relationships among neighbouring frames. We show that our method outperforms other state-of-the-art video enhancement algorithms when trained only on static videos.
arXiv Detail & Related papers (2022-10-09T15:49:46Z)
Look at Adjacent Frames: Video Anomaly Detection without Offline Training [21.334952965297667]
We propose a solution to detect anomalous events in videos without the need to train a model offline. Specifically, our solution is based on a randomly-d multilayer perceptron that is optimized online to reconstruct video frames, pixel-by-pixel, from their frequency information. An incremental learner is used to update parameters of the multilayer perceptron after observing each frame, thus allowing to detect anomalous events along the video stream.
arXiv Detail & Related papers (2022-07-27T21:18:58Z)
Revealing Single Frame Bias for Video-and-Language Learning [115.01000652123882]
We show that a single-frame trained model can achieve better performance than existing methods that use multiple frames for training. This result reveals the existence of a strong "static appearance bias" in popular video-and-language datasets. We propose two new retrieval tasks based on existing fine-grained action recognition datasets that encourage temporal modeling.
arXiv Detail & Related papers (2022-06-07T16:28:30Z)
Efficient Video Segmentation Models with Per-frame Inference [117.97423110566963]
We focus on improving the temporal consistency without introducing overhead in inference. We propose several techniques to learn from the video sequence, including a temporal consistency loss and online/offline knowledge distillation methods.
arXiv Detail & Related papers (2022-02-24T23:51:36Z)
Self-supervised Video Representation Learning Using Inter-intra Contrastive Framework [43.002621928500425]
We propose a self-supervised method to learn feature representations from videos. Because video representation is important, we extend negative samples by introducing intra-negative samples. We conduct experiments on video retrieval and video recognition tasks using the learned video representation.
arXiv Detail & Related papers (2020-08-06T09:08:14Z)
Non-Adversarial Video Synthesis with Learned Priors [53.26777815740381]
We focus on the problem of generating videos from latent noise vectors, without any reference input frames. We develop a novel approach that jointly optimize the input latent space, the weights of a recurrent neural network and a generator through non-adversarial learning. Our approach generates superior quality videos compared to the existing state-of-the-art methods.
arXiv Detail & Related papers (2020-03-21T02:57:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.