Deep Learning-based Image and Video Inpainting: A Survey

• URL: http://arxiv.org/abs/2401.03395v1
• Date: Sun, 7 Jan 2024 05:50:12 GMT
• Title: Deep Learning-based Image and Video Inpainting: A Survey
• Authors: Weize Quan and Jiaxi Chen and Yanli Liu and Dong-Ming Yan and Peter Wonka
• Abstract summary: This paper comprehensively reviews the deep learning-based methods for image and video inpainting. We sort existing methods into different categories from the perspective of their high-level inpainting pipeline. We present evaluation metrics for low-level pixel and high-level perceptional similarity, conduct a performance evaluation, and discuss the strengths and weaknesses of representative inpainting methods.
• Score: 47.53641171826598
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Image and video inpainting is a classic problem in computer vision and computer graphics, aiming to fill in the plausible and realistic content in the missing areas of images and videos. With the advance of deep learning, this problem has achieved significant progress recently. The goal of this paper is to comprehensively review the deep learning-based methods for image and video inpainting. Specifically, we sort existing methods into different categories from the perspective of their high-level inpainting pipeline, present different deep learning architectures, including CNN, VAE, GAN, diffusion models, etc., and summarize techniques for module design. We review the training objectives and the common benchmark datasets. We present evaluation metrics for low-level pixel and high-level perceptional similarity, conduct a performance evaluation, and discuss the strengths and weaknesses of representative inpainting methods. We also discuss related real-world applications. Finally, we discuss open challenges and suggest potential future research directions.

Related papers

• Reimagining Reality: A Comprehensive Survey of Video Inpainting Techniques [6.36998581871295]
  Video inpainting is a process that restores or fills in missing or corrupted portions of video sequences with plausible content. Our study deconstructs major techniques, their underpinning theories, and their effective applications. We employ a human-centric approach to assess visual quality, enlisting a panel of annotators to evaluate the output of different video inpainting techniques.
  arXiv  Detail & Related papers  (2024-01-31T14:41:40Z)
• NPF-200: A Multi-Modal Eye Fixation Dataset and Method for Non-Photorealistic Videos [51.409547544747284]
  NPF-200 is the first large-scale multi-modal dataset of purely non-photorealistic videos with eye fixations. We conduct a series of analyses to gain deeper insights into this task. We propose a universal frequency-aware multi-modal non-photorealistic saliency detection model called NPSNet.
  arXiv  Detail & Related papers  (2023-08-23T14:25:22Z)
• Stroke-based Rendering: From Heuristics to Deep Learning [0.17188280334580194]
  Recent developments in deep learning methods help to bridge the gap between stroke-based paintings and pixel photo generation. We aim to provide a structured introduction and understanding of common challenges and approaches in stroke-based rendering algorithms.
  arXiv  Detail & Related papers  (2022-12-30T05:34:54Z)
• A domain adaptive deep learning solution for scanpath prediction of paintings [66.46953851227454]
  This paper focuses on the eye-movement analysis of viewers during the visual experience of a certain number of paintings. We introduce a new approach to predicting human visual attention, which impacts several cognitive functions for humans. The proposed new architecture ingests images and returns scanpaths, a sequence of points featuring a high likelihood of catching viewers' attention.
  arXiv  Detail & Related papers  (2022-09-22T22:27:08Z)
• Deep Image Deblurring: A Survey [165.32391279761006]
  Deblurring is a classic problem in low-level computer vision, which aims to recover a sharp image from a blurred input image. Recent advances in deep learning have led to significant progress in solving this problem.
  arXiv  Detail & Related papers  (2022-01-26T01:31:30Z)
• Lighting the Darkness in the Deep Learning Era [118.35081853500411]
  Low-light image enhancement (LLIE) aims at improving the perception or interpretability of an image captured in an environment with poor illumination. Recent advances in this area are dominated by deep learning-based solutions. We provide a comprehensive survey to cover various aspects ranging from algorithm taxonomy to unsolved open issues.
  arXiv  Detail & Related papers  (2021-04-21T19:12:19Z)
• Image Segmentation Using Deep Learning: A Survey [58.37211170954998]
  Image segmentation is a key topic in image processing and computer vision. There has been a substantial amount of works aimed at developing image segmentation approaches using deep learning models.
  arXiv  Detail & Related papers  (2020-01-15T21:37:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.