ROMNet: Renovate the Old Memories

• URL: http://arxiv.org/abs/2202.02606v1
• Date: Sat, 5 Feb 2022 17:48:15 GMT
• Title: ROMNet: Renovate the Old Memories
• Authors: Runsheng Xu, Zhengzhong Tu, Yuanqi Du, Xiaoyu Dong, Jinlong Li, Zibo Meng, Jiaqi Ma, Hongkai YU
• Abstract summary: We present a novel reference-based end-to-end learning framework that can jointly repair and colorize degraded legacy pictures. We also create, to our knowledge, the first public and real-world old photo dataset with paired ground truth for evaluating old photo restoration models.
• Score: 25.41639794384076
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Renovating the memories in old photos is an intriguing research topic in computer vision fields. These legacy images often suffer from severe and commingled degradations such as cracks, noise, and color-fading, while lack of large-scale paired old photo datasets makes this restoration task very challenging. In this work, we present a novel reference-based end-to-end learning framework that can jointly repair and colorize the degraded legacy pictures. Specifically, the proposed framework consists of three modules: a restoration sub-network for degradation restoration, a similarity sub-network for color histogram matching and transfer, and a colorization subnet that learns to predict the chroma elements of the images conditioned on chromatic reference signals. The whole system takes advantage of the color histogram priors in a given reference image, which vastly reduces the dependency on large-scale training data. Apart from the proposed method, we also create, to our knowledge, the first public and real-world old photo dataset with paired ground truth for evaluating old photo restoration models, wherein each old photo is paired with a manually restored pristine image by PhotoShop experts. Our extensive experiments conducted on both synthetic and real-world datasets demonstrate that our method significantly outperforms state-of-the-arts both quantitatively and qualitatively.

Related papers

• Preserving Old Memories in Vivid Detail: Human-Interactive Photo Restoration Framework [19.213916633152625]
  Photo restoration can improve the quality of outcomes, but it often comes at a high price in terms of cost and time for restoration. We present the AI-based photo restoration framework composed of multiple stages, where each stage is tailored to enhance and restore specific types of photo damage. We present a novel old photo restoration dataset because we lack a publicly available dataset for our evaluation.
  arXiv  Detail & Related papers  (2024-10-12T13:23:08Z)
• InstructIR: High-Quality Image Restoration Following Human Instructions [61.1546287323136]
  We present the first approach that uses human-written instructions to guide the image restoration model. Our method, InstructIR, achieves state-of-the-art results on several restoration tasks.
  arXiv  Detail & Related papers  (2024-01-29T18:53:33Z)
• Exposure Bracketing is All You Need for Unifying Image Restoration and Enhancement Tasks [50.822601495422916]
  We propose to utilize exposure bracketing photography to unify image restoration and enhancement tasks. Due to the difficulty in collecting real-world pairs, we suggest a solution that first pre-trains the model with synthetic paired data. In particular, a temporally modulated recurrent network (TMRNet) and self-supervised adaptation method are proposed.
  arXiv  Detail & Related papers  (2024-01-01T14:14:35Z)
• All-in-one Multi-degradation Image Restoration Network via Hierarchical Degradation Representation [47.00239809958627]
  We propose a novel All-in-one Multi-degradation Image Restoration Network (AMIRNet) AMIRNet learns a degradation representation for unknown degraded images by progressively constructing a tree structure through clustering. This tree-structured representation explicitly reflects the consistency and discrepancy of various distortions, providing a specific clue for image restoration.
  arXiv  Detail & Related papers  (2023-08-06T04:51:41Z)
• Pik-Fix: Restoring and Colorizing Old Photo [24.366910102387344]
  Restoring and inpainting the visual memories that are present, but often impaired, in old photos remains an intriguing but unsolved research topic. Deep learning presents a plausible avenue, but the lack of large-scale datasets of old photos makes addressing this restoration task very challenging. Here we present a novel reference-based end-to-end learning framework that is able to both repair and colorize old and degraded pictures.
  arXiv  Detail & Related papers  (2022-05-04T05:46:43Z)
• Detecting Recolored Image by Spatial Correlation [60.08643417333974]
  Image recoloring is an emerging editing technique that can manipulate the color values of an image to give it a new style. In this paper, we explore a solution from the perspective of the spatial correlation, which exhibits the generic detection capability for both conventional and deep learning-based recoloring. Our method achieves the state-of-the-art detection accuracy on multiple benchmark datasets and exhibits well generalization for unknown types of recoloring methods.
  arXiv  Detail & Related papers  (2022-04-23T01:54:06Z)
• Old Photo Restoration via Deep Latent Space Translation [46.73615925108932]
  We propose to restore old photos that suffer from severe degradation through a deep learning approach. The degradation in real photos is complex and the domain gap between synthetic images and real old photos makes the network fail to generalize. Specifically, we train two variational autoencoders (VAEs) to respectively transform old photos and clean photos into two latent spaces. And the translation between these two latent spaces is learned with synthetic paired data.
  arXiv  Detail & Related papers  (2020-09-14T08:51:53Z)
• Bringing Old Photos Back to Life [46.73615925108932]
  The degradation in real photos is complex and the domain gap between synthetic images and real old photos makes the network fail to generalize. We propose a novel triplet domain translation network by leveraging real photos along with massive synthetic image pairs. The proposed method outperforms state-of-the-art methods in terms of visual quality for old photos restoration.
  arXiv  Detail & Related papers  (2020-04-20T17:59:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.