Blind Video Deflickering by Neural Filtering with a Flawed Atlas
- URL: http://arxiv.org/abs/2303.08120v1
- Date: Tue, 14 Mar 2023 17:52:29 GMT
- Title: Blind Video Deflickering by Neural Filtering with a Flawed Atlas
- Authors: Chenyang Lei, Xuanchi Ren, Zhaoxiang Zhang, Qifeng Chen
- Abstract summary: We propose a general flicker removal framework that only receives a single flickering video as input without additional guidance.
The core of our approach is utilizing the neural atlas in cooperation with a neural filtering strategy.
To validate our method, we construct a dataset that contains diverse real-world flickering videos.
- Score: 90.96203200658667
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Many videos contain flickering artifacts. Common causes of flicker include
video processing algorithms, video generation algorithms, and capturing videos
under specific situations. Prior work usually requires specific guidance such
as the flickering frequency, manual annotations, or extra consistent videos to
remove the flicker. In this work, we propose a general flicker removal
framework that only receives a single flickering video as input without
additional guidance. Since it is blind to a specific flickering type or
guidance, we name this "blind deflickering." The core of our approach is
utilizing the neural atlas in cooperation with a neural filtering strategy. The
neural atlas is a unified representation for all frames in a video that
provides temporal consistency guidance but is flawed in many cases. To this
end, a neural network is trained to mimic a filter to learn the consistent
features (e.g., color, brightness) and avoid introducing the artifacts in the
atlas. To validate our method, we construct a dataset that contains diverse
real-world flickering videos. Extensive experiments show that our method
achieves satisfying deflickering performance and even outperforms baselines
that use extra guidance on a public benchmark.
Related papers
- Look at Adjacent Frames: Video Anomaly Detection without Offline
Training [21.334952965297667]
We propose a solution to detect anomalous events in videos without the need to train a model offline.
Specifically, our solution is based on a randomly-d multilayer perceptron that is optimized online to reconstruct video frames, pixel-by-pixel, from their frequency information.
An incremental learner is used to update parameters of the multilayer perceptron after observing each frame, thus allowing to detect anomalous events along the video stream.
arXiv Detail & Related papers (2022-07-27T21:18:58Z) - Deep Video Prior for Video Consistency and Propagation [58.250209011891904]
We present a novel and general approach for blind video temporal consistency.
Our method is only trained on a pair of original and processed videos directly instead of a large dataset.
We show that temporal consistency can be achieved by training a convolutional neural network on a video with Deep Video Prior.
arXiv Detail & Related papers (2022-01-27T16:38:52Z) - Video Salient Object Detection via Contrastive Features and Attention
Modules [106.33219760012048]
We propose a network with attention modules to learn contrastive features for video salient object detection.
A co-attention formulation is utilized to combine the low-level and high-level features.
We show that the proposed method requires less computation, and performs favorably against the state-of-the-art approaches.
arXiv Detail & Related papers (2021-11-03T17:40:32Z) - Highlight Timestamp Detection Model for Comedy Videos via Multimodal
Sentiment Analysis [1.6181085766811525]
We propose a multimodal structure to obtain state-of-the-art performance in this field.
We select several benchmarks for multimodal video understanding and apply the most suitable model to find the best performance.
arXiv Detail & Related papers (2021-05-28T08:39:19Z) - Less is More: ClipBERT for Video-and-Language Learning via Sparse
Sampling [98.41300980759577]
A canonical approach to video-and-language learning dictates a neural model to learn from offline-extracted dense video features.
We propose a generic framework ClipBERT that enables affordable end-to-end learning for video-and-language tasks.
Experiments on text-to-video retrieval and video question answering on six datasets demonstrate that ClipBERT outperforms existing methods.
arXiv Detail & Related papers (2021-02-11T18:50:16Z) - Blind Video Temporal Consistency via Deep Video Prior [61.062900556483164]
We present a novel and general approach for blind video temporal consistency.
Our method is only trained on a pair of original and processed videos directly.
We show that temporal consistency can be achieved by training a convolutional network on a video with the Deep Video Prior.
arXiv Detail & Related papers (2020-10-22T16:19:20Z) - Self-supervised Video Representation Learning by Pace Prediction [48.029602040786685]
This paper addresses the problem of self-supervised video representation learning from a new perspective -- by video pace prediction.
It stems from the observation that human visual system is sensitive to video pace.
We randomly sample training clips in different paces and ask a neural network to identify the pace for each video clip.
arXiv Detail & Related papers (2020-08-13T12:40:24Z) - Adversarially Robust Frame Sampling with Bounded Irregularities [11.434633941880143]
Video analysis tools for automatically extracting meaningful information from videos are widely studied and deployed.
Most of them use deep neural networks which are computationally expensive, feeding only a subset of video frames into such algorithms is desired.
We present an elegant solution to this sampling problem that is provably robust against adversarial attacks and introduces bounded irregularities as well.
arXiv Detail & Related papers (2020-02-04T06:33:43Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.