DeepRemaster: Temporal Source-Reference Attention Networks for Comprehensive Video Enhancement

• URL: http://arxiv.org/abs/2009.08692v1
• Date: Fri, 18 Sep 2020 08:55:11 GMT
• Title: DeepRemaster: Temporal Source-Reference Attention Networks for Comprehensive Video Enhancement
• Authors: Satoshi Iizuka and Edgar Simo-Serra
• Abstract summary: We propose a framework to tackle the entire remastering task semi-interactively. Our work is based on temporal convolutional neural networks with attention mechanisms trained on videos with data-driven deterioration simulation.
• Score: 32.679447725129165
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The remastering of vintage film comprises of a diversity of sub-tasks including super-resolution, noise removal, and contrast enhancement which aim to restore the deteriorated film medium to its original state. Additionally, due to the technical limitations of the time, most vintage film is either recorded in black and white, or has low quality colors, for which colorization becomes necessary. In this work, we propose a single framework to tackle the entire remastering task semi-interactively. Our work is based on temporal convolutional neural networks with attention mechanisms trained on videos with data-driven deterioration simulation. Our proposed source-reference attention allows the model to handle an arbitrary number of reference color images to colorize long videos without the need for segmentation while maintaining temporal consistency. Quantitative analysis shows that our framework outperforms existing approaches, and that, in contrast to existing approaches, the performance of our framework increases with longer videos and more reference color images.

Related papers

• VanGogh: A Unified Multimodal Diffusion-based Framework for Video Colorization [53.35016574938809]
  Video colorization aims to transform grayscale videos into vivid color representations while maintaining temporal consistency and structural integrity. Existing video colorization methods often suffer from color bleeding and lack comprehensive control. We introduce VanGogh, a unified multimodal diffusion-based framework for video colorization.
  arXiv  Detail & Related papers  (2025-01-16T12:20:40Z)
• SVFR: A Unified Framework for Generalized Video Face Restoration [86.17060212058452]
  Face Restoration (FR) is a crucial area within image and video processing, focusing on reconstructing high-quality portraits from degraded inputs. We propose a novel approach for the Generalized Video Face Restoration task, which integrates video BFR, inpainting, and colorization tasks. This work advances the state-of-the-art in video FR and establishes a new paradigm for generalized video face restoration.
  arXiv  Detail & Related papers  (2025-01-02T12:51:20Z)
• LVCD: Reference-based Lineart Video Colorization with Diffusion Models [18.0983825973013]
  We propose the first video diffusion framework for reference-based lineart video colorization. We leverage a large-scale pretrained video diffusion model to generate colorized animation videos. Our method is capable of generating high-quality, long temporal-consistent animation videos.
  arXiv  Detail & Related papers  (2024-09-19T17:59:48Z)
• LatentColorization: Latent Diffusion-Based Speaker Video Colorization [1.2641141743223379]
  We introduce a novel solution for achieving temporal consistency in video colorization. We demonstrate strong improvements on established image quality metrics compared to other existing methods. Our dataset encompasses a combination of conventional datasets and videos from television/movies.
  arXiv  Detail & Related papers  (2024-05-09T12:06:06Z)
• FlowChroma -- A Deep Recurrent Neural Network for Video Colorization [1.0499611180329804]
  We develop an automated video colorization framework that minimizes the flickering of colors across frames. We show that recurrent neural networks can be successfully used to improve color consistency in video colorization.
  arXiv  Detail & Related papers  (2023-05-23T05:41:53Z)
• Temporal Consistent Automatic Video Colorization via Semantic Correspondence [12.107878178519128]
  We propose a novel video colorization framework, which combines semantic correspondence into automatic video colorization. In the NTIRE 2023 Video Colorization Challenge, our method ranks at the 3rd place in Color Distribution Consistency (CDC) Optimization track.
  arXiv  Detail & Related papers  (2023-05-13T12:06:09Z)
• Temporally Consistent Video Colorization with Deep Feature Propagation and Self-regularization Learning [90.38674162878496]
  We propose a novel temporally consistent video colorization framework (TCVC) TCVC effectively propagates frame-level deep features in a bidirectional way to enhance the temporal consistency of colorization. Experiments demonstrate that our method can not only obtain visually pleasing colorized video, but also achieve clearly better temporal consistency than state-of-the-art methods.
  arXiv  Detail & Related papers  (2021-10-09T13:00:14Z)
• Restoration of Video Frames from a Single Blurred Image with Motion Understanding [69.90724075337194]
  We propose a novel framework to generate clean video frames from a single motion-red image. We formulate video restoration from a single blurred image as an inverse problem by setting clean image sequence and their respective motion as latent factors. Our framework is based on anblur-decoder structure with spatial transformer network modules.
  arXiv  Detail & Related papers  (2021-04-19T08:32:57Z)
• An Efficient Recurrent Adversarial Framework for Unsupervised Real-Time Video Enhancement [132.60976158877608]
  We propose an efficient adversarial video enhancement framework that learns directly from unpaired video examples. In particular, our framework introduces new recurrent cells that consist of interleaved local and global modules for implicit integration of spatial and temporal information. The proposed design allows our recurrent cells to efficiently propagate-temporal-information across frames and reduces the need for high complexity networks.
  arXiv  Detail & Related papers  (2020-12-24T00:03:29Z)
• Learning Joint Spatial-Temporal Transformations for Video Inpainting [58.939131620135235]
  We propose to learn a joint Spatial-Temporal Transformer Network (STTN) for video inpainting. We simultaneously fill missing regions in all input frames by self-attention, and propose to optimize STTN by a spatial-temporal adversarial loss.
  arXiv  Detail & Related papers  (2020-07-20T16:35:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.