Deep image prior inpainting of ancient frescoes in the Mediterranean Alpine arc

• URL: http://arxiv.org/abs/2306.14209v2
• Date: Mon, 11 Dec 2023 15:30:01 GMT
• Title: Deep image prior inpainting of ancient frescoes in the Mediterranean Alpine arc
• Authors: Fabio Merizzi, Perrine Saillard, Oceane Acquier, Elena Morotti, Elena Loli Piccolomini, Luca Calatroni and Rosa Maria Dess\`i
• Abstract summary: DIP-based inpainting reduces artefacts and better adapts to contextual/non-local information, thus providing a valuable tool for art historians. We apply such approach to reconstruct missing image contents in a dataset of highly damaged digital images of medieval paintings located into several chapels in the Mediterranean Alpine Arc.
• Score: 0.3958317527488534
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: The unprecedented success of image reconstruction approaches based on deep neural networks has revolutionised both the processing and the analysis paradigms in several applied disciplines. In the field of digital humanities, the task of digital reconstruction of ancient frescoes is particularly challenging due to the scarce amount of available training data caused by ageing, wear, tear and retouching over time. To overcome these difficulties, we consider the Deep Image Prior (DIP) inpainting approach which computes appropriate reconstructions by relying on the progressive updating of an untrained convolutional neural network so as to match the reliable piece of information in the image at hand while promoting regularisation elsewhere. In comparison with state-of-the-art approaches (based on variational/PDEs and patch-based methods), DIP-based inpainting reduces artefacts and better adapts to contextual/non-local information, thus providing a valuable and effective tool for art historians. As a case study, we apply such approach to reconstruct missing image contents in a dataset of highly damaged digital images of medieval paintings located into several chapels in the Mediterranean Alpine Arc and provide a detailed description on how visible and invisible (e.g., infrared) information can be integrated for identifying and reconstructing damaged image regions.

Related papers

• All-in-one Multi-degradation Image Restoration Network via Hierarchical Degradation Representation [47.00239809958627]
  We propose a novel All-in-one Multi-degradation Image Restoration Network (AMIRNet) AMIRNet learns a degradation representation for unknown degraded images by progressively constructing a tree structure through clustering. This tree-structured representation explicitly reflects the consistency and discrepancy of various distortions, providing a specific clue for image restoration.
  arXiv  Detail & Related papers  (2023-08-06T04:51:41Z)
• Wide & deep learning for spatial & intensity adaptive image restoration [16.340992967330603]
  We propose an ingenious and efficient multi-frame image restoration network (DparNet) with wide & deep architecture. The degradation prior is directly learned from degraded images in form of key degradation parameter matrix. The wide & deep architecture in DparNet enables the learned parameters to directly modulate the final restoring results.
  arXiv  Detail & Related papers  (2023-05-30T03:24:09Z)
• Structure-guided Image Outpainting [2.7215474244966296]
  Image outpainting has difficulties caused by large-scale area loss and less legitimate neighboring information. We propose a deep learning method based on Generative Adrial Network (GAN) and condition edges as structural prior. The newly added semantic embedding loss is proved effective in practice.
  arXiv  Detail & Related papers  (2022-12-21T20:24:24Z)
• A Survey of Deep Face Restoration: Denoise, Super-Resolution, Deblur, Artifact Removal [177.21001709272144]
  Face Restoration (FR) aims to restore High-Quality (HQ) faces from Low-Quality (LQ) input images. This paper comprehensively surveys recent advances in deep learning techniques for face restoration.
  arXiv  Detail & Related papers  (2022-11-05T07:08:15Z)
• A Deep Generative Approach to Oversampling in Ptychography [9.658250977094562]
  A major drawback of ptychography is the long data acquisition time. We propose complementing sparsely acquired or undersampled data with data sampled from a deep generative network. Because the deep generative network is pre-trained and its output can be computed as we collect data, the experimental data and the time to acquire the data can be reduced.
  arXiv  Detail & Related papers  (2022-07-28T22:02:01Z)
• Variational Deep Image Restoration [20.195082841065947]
  This paper presents a new variational inference framework for image restoration and a convolutional neural network (CNN) structure that can solve the restoration problems described by the proposed framework. Specifically, our method delivers state-of-the-art performance on Gaussian denoising, real-world noise reduction, blind image super-resolution, and JPEG compression artifacts reduction.
  arXiv  Detail & Related papers  (2022-07-03T16:32:15Z)
• Unsupervised Restoration of Weather-affected Images using Deep Gaussian Process-based CycleGAN [92.15895515035795]
  We describe an approach for supervising deep networks that are based on CycleGAN. We introduce new losses for training CycleGAN that lead to more effective training, resulting in high-quality reconstructions. We demonstrate that the proposed method can be effectively applied to different restoration tasks like de-raining, de-hazing and de-snowing.
  arXiv  Detail & Related papers  (2022-04-23T01:30:47Z)
• Deep Image Deblurring: A Survey [165.32391279761006]
  Deblurring is a classic problem in low-level computer vision, which aims to recover a sharp image from a blurred input image. Recent advances in deep learning have led to significant progress in solving this problem.
  arXiv  Detail & Related papers  (2022-01-26T01:31:30Z)
• Is Deep Image Prior in Need of a Good Education? [57.3399060347311]
  Deep image prior was introduced as an effective prior for image reconstruction. Despite its impressive reconstructive properties, the approach is slow when compared to learned or traditional reconstruction techniques. We develop a two-stage learning paradigm to address the computational challenge.
  arXiv  Detail & Related papers  (2021-11-23T15:08:26Z)
• Underwater Image Restoration via Contrastive Learning and a Real-world Dataset [59.35766392100753]
  We present a novel method for underwater image restoration based on unsupervised image-to-image translation framework. Our proposed method leveraged contrastive learning and generative adversarial networks to maximize the mutual information between raw and restored images.
  arXiv  Detail & Related papers  (2021-06-20T16:06:26Z)
• Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation [181.08127307338654]
  This work presents an effective way to exploit the image prior captured by a generative adversarial network (GAN) trained on large-scale natural images. The deep generative prior (DGP) provides compelling results to restore missing semantics, e.g., color, patch, resolution, of various degraded images.
  arXiv  Detail & Related papers  (2020-03-30T17:45:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.