Can Artificial Intelligence Reconstruct Ancient Mosaics?

• URL: http://arxiv.org/abs/2210.06145v1
• Date: Fri, 7 Oct 2022 19:42:09 GMT
• Title: Can Artificial Intelligence Reconstruct Ancient Mosaics?
• Authors: Fernando Moral-Andr\'es and Elena Merino-G\'omez and Pedro Reviriego and Fabrizio Lombardi
• Abstract summary: In the last years, Artificial Intelligence (AI) has made impressive progress in the generation of images from text descriptions and reference images. In this paper, we explore whether this innovative technology can be used to reconstruct mosaics with missing parts. Results are promising showing that AI is able to interpret the key features of the mosaics and is able to produce reconstructions that capture the essence of the scene.
• Score: 71.93546109923456
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: A large number of ancient mosaics have not reached us because they have been destroyed by erosion, earthquakes, looting or even used as materials in newer construction. To make things worse, among the small fraction of mosaics that we have been able to recover, many are damaged or incomplete. Therefore, restoration and reconstruction of mosaics play a fundamental role to preserve cultural heritage and to understand the role of mosaics in ancient cultures. This reconstruction has traditionally been done manually and more recently using computer graphics programs but always by humans. In the last years, Artificial Intelligence (AI) has made impressive progress in the generation of images from text descriptions and reference images. State of the art AI tools such as DALL-E2 can generate high quality images from text prompts and can take a reference image to guide the process. In august 2022, DALL-E2 launched a new feature called outpainting that takes as input an incomplete image and a text prompt and then generates a complete image filling the missing parts. In this paper, we explore whether this innovative technology can be used to reconstruct mosaics with missing parts. Hence a set of ancient mosaics have been used and reconstructed using DALL-E2; results are promising showing that AI is able to interpret the key features of the mosaics and is able to produce reconstructions that capture the essence of the scene. However, in some cases AI fails to reproduce some details, geometric forms or introduces elements that are not consistent with the rest of the mosaic. This suggests that as AI image generation technology matures in the next few years, it could be a valuable tool for mosaic reconstruction going forward.

Related papers

• Zero-Shot Detection of AI-Generated Images [54.01282123570917]
  We propose a zero-shot entropy-based detector (ZED) to detect AI-generated images. Inspired by recent works on machine-generated text detection, our idea is to measure how surprising the image under analysis is compared to a model of real images. ZED achieves an average improvement of more than 3% over the SoTA in terms of accuracy.
  arXiv  Detail & Related papers  (2024-09-24T08:46:13Z)
• Restoring Ancient Ideograph: A Multimodal Multitask Neural Network Approach [11.263700269889654]
  This paper proposes a novel Multimodal Multitask Restoring Model (MMRM) to restore ancient texts. It combines context understanding with residual visual information from damaged ancient artefacts, enabling it to predict damaged characters and generate restored images simultaneously.
  arXiv  Detail & Related papers  (2024-03-11T12:57:28Z)
• InstructIR: High-Quality Image Restoration Following Human Instructions [61.1546287323136]
  We present the first approach that uses human-written instructions to guide the image restoration model. Our method, InstructIR, achieves state-of-the-art results on several restoration tasks.
  arXiv  Detail & Related papers  (2024-01-29T18:53:33Z)
• Generative AI and the History of Architecture [0.6993026261767287]
  We investigate generative AI platforms for text and image generation for different architectural styles. We analyze a data set of 101 million Midjourney queries to see if and how practitioners are already querying for specific architectural concepts.
  arXiv  Detail & Related papers  (2023-12-22T23:02:09Z)
• Knowledge-Aware Artifact Image Synthesis with LLM-Enhanced Prompting and Multi-Source Supervision [5.517240672957627]
  We propose a novel knowledge-aware artifact image synthesis approach that brings lost historical objects accurately into their visual forms. Compared to existing approaches, our proposed model produces higher-quality artifact images that align better with the implicit details and historical knowledge contained within written documents.
  arXiv  Detail & Related papers  (2023-12-13T11:03:07Z)
• Adaptation of the super resolution SOTA for Art Restoration in camera capture images [0.0]
  We adapt the current state-of-art for the image super-resolution based on the Diffusion Model (DM) and fine-tune it for Image art restoration. Our results show that instead of fine-tunning multiple different models for different kinds of degradation, fine-tuning one super-resolution.
  arXiv  Detail & Related papers  (2023-09-24T14:47:29Z)
• Create Your World: Lifelong Text-to-Image Diffusion [75.14353789007902]
  We propose Lifelong text-to-image Diffusion Model (L2DM) to overcome knowledge "catastrophic forgetting" for the past encountered concepts. In respect of knowledge "catastrophic forgetting", our L2DM framework devises a task-aware memory enhancement module and a elastic-concept distillation module. Our model can generate more faithful image across a range of continual text prompts in terms of both qualitative and quantitative metrics.
  arXiv  Detail & Related papers  (2023-09-08T16:45:56Z)
• Unfinished Architectures: A Perspective from Artificial Intelligence [73.52315464582637]
  Development of Artificial Intelligence (AI) opens new avenues for the proposal of possibilities for the completion of unfinished architectures. Recent appearance of tools such as DALL-E, capable of completing images guided by a textual description. In this article we explore the use of these new AI tools for the completion of unfinished facades of historical temples and analyse the still germinal stadium in the field of architectural graphic composition.
  arXiv  Detail & Related papers  (2023-03-03T13:05:10Z)
• Fully Context-Aware Image Inpainting with a Learned Semantic Pyramid [102.24539566851809]
  Restoring reasonable and realistic content for arbitrary missing regions in images is an important yet challenging task. Recent image inpainting models have made significant progress in generating vivid visual details, but they can still lead to texture blurring or structural distortions. We propose the Semantic Pyramid Network (SPN) motivated by the idea that learning multi-scale semantic priors can greatly benefit the recovery of locally missing content in images.
  arXiv  Detail & Related papers  (2021-12-08T04:33:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.