Pik-Fix: Restoring and Colorizing Old Photo
- URL: http://arxiv.org/abs/2205.01902v1
- Date: Wed, 4 May 2022 05:46:43 GMT
- Title: Pik-Fix: Restoring and Colorizing Old Photo
- Authors: Runsheng Xu, Zhengzhong Tu, Yuanqi Du, Xiaoyu Dong, Jinlong Li, Zibo
Meng, Jiaqi Ma, Alan Bovik, Hongkai Yu
- Abstract summary: Restoring and inpainting the visual memories that are present, but often impaired, in old photos remains an intriguing but unsolved research topic.
Deep learning presents a plausible avenue, but the lack of large-scale datasets of old photos makes addressing this restoration task very challenging.
Here we present a novel reference-based end-to-end learning framework that is able to both repair and colorize old and degraded pictures.
- Score: 24.366910102387344
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: Restoring and inpainting the visual memories that are present, but often
impaired, in old photos remains an intriguing but unsolved research topic.
Decades-old photos often suffer from severe and commingled degradation such as
cracks, defocus, and color-fading, which are difficult to treat individually
and harder to repair when they interact. Deep learning presents a plausible
avenue, but the lack of large-scale datasets of old photos makes addressing
this restoration task very challenging. Here we present a novel reference-based
end-to-end learning framework that is able to both repair and colorize old and
degraded pictures. Our proposed framework consists of three modules: a
restoration sub-network that conducts restoration from degradations, a
similarity sub-network that performs color histogram matching and color
transfer, and a colorization subnet that learns to predict the chroma elements
of images that have been conditioned on chromatic reference signals. The
overall system makes uses of color histogram priors from reference images,
which greatly reduces the need for large-scale training data. We have also
created a first-of-a-kind public dataset of real old photos that are paired
with ground truth "pristine" photos that have been that have been manually
restored by PhotoShop experts. We conducted extensive experiments on this
dataset and synthetic datasets, and found that our method significantly
outperforms previous state-of-the-art models using both qualitative comparisons
and quantitative measurements.
Related papers
- Preserving Old Memories in Vivid Detail: Human-Interactive Photo Restoration Framework [19.213916633152625]
Photo restoration can improve the quality of outcomes, but it often comes at a high price in terms of cost and time for restoration.
We present the AI-based photo restoration framework composed of multiple stages, where each stage is tailored to enhance and restore specific types of photo damage.
We present a novel old photo restoration dataset because we lack a publicly available dataset for our evaluation.
arXiv Detail & Related papers (2024-10-12T13:23:08Z) - InstructIR: High-Quality Image Restoration Following Human Instructions [61.1546287323136]
We present the first approach that uses human-written instructions to guide the image restoration model.
Our method, InstructIR, achieves state-of-the-art results on several restoration tasks.
arXiv Detail & Related papers (2024-01-29T18:53:33Z) - Exposure Bracketing is All You Need for Unifying Image Restoration and Enhancement Tasks [50.822601495422916]
We propose to utilize exposure bracketing photography to unify image restoration and enhancement tasks.
Due to the difficulty in collecting real-world pairs, we suggest a solution that first pre-trains the model with synthetic paired data.
In particular, a temporally modulated recurrent network (TMRNet) and self-supervised adaptation method are proposed.
arXiv Detail & Related papers (2024-01-01T14:14:35Z) - Invertible Rescaling Network and Its Extensions [118.72015270085535]
In this work, we propose a novel invertible framework to model the bidirectional degradation and restoration from a new perspective.
We develop invertible models to generate valid degraded images and transform the distribution of lost contents.
Then restoration is made tractable by applying the inverse transformation on the generated degraded image together with a randomly-drawn latent variable.
arXiv Detail & Related papers (2022-10-09T06:58:58Z) - Detecting Recolored Image by Spatial Correlation [60.08643417333974]
Image recoloring is an emerging editing technique that can manipulate the color values of an image to give it a new style.
In this paper, we explore a solution from the perspective of the spatial correlation, which exhibits the generic detection capability for both conventional and deep learning-based recoloring.
Our method achieves the state-of-the-art detection accuracy on multiple benchmark datasets and exhibits well generalization for unknown types of recoloring methods.
arXiv Detail & Related papers (2022-04-23T01:54:06Z) - ROMNet: Renovate the Old Memories [25.41639794384076]
We present a novel reference-based end-to-end learning framework that can jointly repair and colorize degraded legacy pictures.
We also create, to our knowledge, the first public and real-world old photo dataset with paired ground truth for evaluating old photo restoration models.
arXiv Detail & Related papers (2022-02-05T17:48:15Z) - BIGPrior: Towards Decoupling Learned Prior Hallucination and Data
Fidelity in Image Restoration [14.34815548338413]
We present an approach with decoupled network-prior based hallucination and data fidelity terms.
We use network inversion to extract image prior information from a generative network.
Our method, though partly reliant on the quality of the generative network inversion, is competitive with state-of-the-art supervised and task-specific restoration methods.
arXiv Detail & Related papers (2020-11-03T01:16:41Z) - Image Colorization: A Survey and Dataset [94.59768013860668]
This article presents a comprehensive survey of state-of-the-art deep learning-based image colorization techniques.
It categorizes the existing colorization techniques into seven classes and discusses important factors governing their performance.
We perform an extensive experimental evaluation of existing image colorization methods using both existing datasets and our proposed one.
arXiv Detail & Related papers (2020-08-25T01:22:52Z) - Instance-aware Image Colorization [51.12040118366072]
In this paper, we propose a method for achieving instance-aware colorization.
Our network architecture leverages an off-the-shelf object detector to obtain cropped object images.
We use a similar network to extract the full-image features and apply a fusion module to predict the final colors.
arXiv Detail & Related papers (2020-05-21T17:59:23Z) - Bringing Old Photos Back to Life [46.73615925108932]
The degradation in real photos is complex and the domain gap between synthetic images and real old photos makes the network fail to generalize.
We propose a novel triplet domain translation network by leveraging real photos along with massive synthetic image pairs.
The proposed method outperforms state-of-the-art methods in terms of visual quality for old photos restoration.
arXiv Detail & Related papers (2020-04-20T17:59:23Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.