Bringing Old Films Back to Life
- URL: http://arxiv.org/abs/2203.17276v1
- Date: Thu, 31 Mar 2022 17:59:59 GMT
- Title: Bringing Old Films Back to Life
- Authors: Ziyu Wan and Bo Zhang and Dongdong Chen and Jing Liao
- Abstract summary: We present a learning-based framework, recurrent transformer network (RTN), to restore heavily degraded old films.
Our method is based on the hidden knowledge learned from adjacent frames that contain abundant information about the occlusion.
- Score: 33.78936333249432
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: We present a learning-based framework, recurrent transformer network (RTN),
to restore heavily degraded old films. Instead of performing frame-wise
restoration, our method is based on the hidden knowledge learned from adjacent
frames that contain abundant information about the occlusion, which is
beneficial to restore challenging artifacts of each frame while ensuring
temporal coherency. Moreover, contrasting the representation of the current
frame and the hidden knowledge makes it possible to infer the scratch position
in an unsupervised manner, and such defect localization generalizes well to
real-world degradations. To better resolve mixed degradation and compensate for
the flow estimation error during frame alignment, we propose to leverage more
expressive transformer blocks for spatial restoration. Experiments on both
synthetic dataset and real-world old films demonstrate the significant
superiority of the proposed RTN over existing solutions. In addition, the same
framework can effectively propagate the color from keyframes to the whole
video, ultimately yielding compelling restored films. The implementation and
model will be released at
https://github.com/raywzy/Bringing-Old-Films-Back-to-Life.
Related papers
- Learning to Remove Wrinkled Transparent Film with Polarized Prior [33.56569839317172]
Film Removal (FR) attempts to remove the interference of wrinkled transparent films and reconstruct the original information under films for industrial recognition systems.
We build a practical dataset with polarization information containing paired data with and without transparent film.
We aim to remove interference from the film (specular highlights and other degradations) with an end-to-end framework.
arXiv Detail & Related papers (2024-03-07T09:56:56Z) - Video Dynamics Prior: An Internal Learning Approach for Robust Video
Enhancements [83.5820690348833]
We present a framework for low-level vision tasks that does not require any external training data corpus.
Our approach learns neural modules by optimizing over a corrupted sequence, leveraging the weights of the coherence-temporal test and statistics internal statistics.
arXiv Detail & Related papers (2023-12-13T01:57:11Z) - Dual-Stream Knowledge-Preserving Hashing for Unsupervised Video
Retrieval [67.52910255064762]
We design a simple dual-stream structure, including a temporal layer and a hash layer.
We first design a simple dual-stream structure, including a temporal layer and a hash layer.
With the help of semantic similarity knowledge obtained from self-supervision, the hash layer learns to capture information for semantic retrieval.
In this way, the model naturally preserves the disentangled semantics into binary codes.
arXiv Detail & Related papers (2023-10-12T03:21:12Z) - RIGID: Recurrent GAN Inversion and Editing of Real Face Videos [73.97520691413006]
GAN inversion is indispensable for applying the powerful editability of GAN to real images.
Existing methods invert video frames individually often leading to undesired inconsistent results over time.
We propose a unified recurrent framework, named textbfRecurrent vtextbfIdeo textbfGAN textbfInversion and etextbfDiting (RIGID)
Our framework learns the inherent coherence between input frames in an end-to-end manner.
arXiv Detail & Related papers (2023-08-11T12:17:24Z) - Making Reconstruction-based Method Great Again for Video Anomaly
Detection [64.19326819088563]
Anomaly detection in videos is a significant yet challenging problem.
Existing reconstruction-based methods rely on old-fashioned convolutional autoencoders.
We propose a new autoencoder model for enhanced consecutive frame reconstruction.
arXiv Detail & Related papers (2023-01-28T01:57:57Z) - Restoration of User Videos Shared on Social Media [27.16457737969977]
User videos shared on social media platforms usually suffer from degradations caused by unknown proprietary processing procedures.
This paper presents a new general video restoration framework for the restoration of user videos shared on social media platforms.
In contrast to most deep learning-based video restoration methods that perform end-to-end mapping, our new method, Video restOration through adapTive dEgradation Sensing (VOTES), introduces the concept of a degradation feature map (DFM) to explicitly guide the video restoration process.
arXiv Detail & Related papers (2022-08-18T02:28:43Z) - A Simple Baseline for Video Restoration with Grouped Spatial-temporal
Shift [36.71578909392314]
In this study, we propose a simple yet effective framework for video restoration.
Our approach is based on grouped spatial-temporal shift, which is a lightweight and straightforward technique.
Our framework outperforms the previous state-of-the-art method, while using less than a quarter of its computational cost.
arXiv Detail & Related papers (2022-06-22T02:16:47Z) - Recurrent Video Restoration Transformer with Guided Deformable Attention [116.1684355529431]
We propose RVRT, which processes local neighboring frames in parallel within a globally recurrent framework.
RVRT achieves state-of-the-art performance on benchmark datasets with balanced model size, testing memory and runtime.
arXiv Detail & Related papers (2022-06-05T10:36:09Z) - VRT: A Video Restoration Transformer [126.79589717404863]
Video restoration (e.g., video super-resolution) aims to restore high-quality frames from low-quality frames.
We propose a Video Restoration Transformer (VRT) with parallel frame prediction and long-range temporal dependency modelling abilities.
arXiv Detail & Related papers (2022-01-28T17:54:43Z) - Generative Memorize-Then-Recall framework for low bit-rate Surveillance
Video Compression [29.716388163447345]
Surveillance video feature (memory) for Group of Picture (GoP) and skeleton for each frame (clue)
memory is obtained by sequentially feeding frame inside GoP into a recurrent neural coding, describing appearance for objects that appeared inside GoP.
Experiments indicate that our method effectively generates realistic reconstruction based on appearance and skeleton.
arXiv Detail & Related papers (2019-12-30T08:34:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.