Reimagining Reality: A Comprehensive Survey of Video Inpainting Techniques

• URL: http://arxiv.org/abs/2401.17883v1
• Date: Wed, 31 Jan 2024 14:41:40 GMT
• Title: Reimagining Reality: A Comprehensive Survey of Video Inpainting Techniques
• Authors: Shreyank N Gowda, Yash Thakre, Shashank Narayana Gowda, Xiaobo Jin
• Abstract summary: Video inpainting is a process that restores or fills in missing or corrupted portions of video sequences with plausible content. Our study deconstructs major techniques, their underpinning theories, and their effective applications. We employ a human-centric approach to assess visual quality, enlisting a panel of annotators to evaluate the output of different video inpainting techniques.
• Score: 6.36998581871295
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper offers a comprehensive analysis of recent advancements in video inpainting techniques, a critical subset of computer vision and artificial intelligence. As a process that restores or fills in missing or corrupted portions of video sequences with plausible content, video inpainting has evolved significantly with the advent of deep learning methodologies. Despite the plethora of existing methods and their swift development, the landscape remains complex, posing challenges to both novices and established researchers. Our study deconstructs major techniques, their underpinning theories, and their effective applications. Moreover, we conduct an exhaustive comparative study, centering on two often-overlooked dimensions: visual quality and computational efficiency. We adopt a human-centric approach to assess visual quality, enlisting a panel of annotators to evaluate the output of different video inpainting techniques. This provides a nuanced qualitative understanding that complements traditional quantitative metrics. Concurrently, we delve into the computational aspects, comparing inference times and memory demands across a standardized hardware setup. This analysis underscores the balance between quality and efficiency: a critical consideration for practical applications where resources may be constrained. By integrating human validation and computational resource comparison, this survey not only clarifies the present landscape of video inpainting techniques but also charts a course for future explorations in this vibrant and evolving field.

Related papers

• Perceptually Optimized Super Resolution [7.728090438152828]
  We propose a perceptually inspired and architecture-agnostic approach for controlling the visual quality and efficiency of super-resolution techniques. The core is a perceptual model that dynamically guides super-resolution methods according to the human's sensitivity to image details. We demonstrate the application of our proposed model in combination with network branching, and network complexity reduction to improve the computational efficiency of super-resolution methods without visible quality loss.
  arXiv  Detail & Related papers  (2024-11-26T15:24:45Z)
• Deep video representation learning: a survey [4.9589745881431435]
  We recent sequential feature learning methods for visual data and compare their pros and cons for general video analysis. Building effective features for videos is a fundamental problem in computer vision tasks involving video analysis and understanding.
  arXiv  Detail & Related papers  (2024-05-10T16:20:11Z)
• Deep Learning-based Image and Video Inpainting: A Survey [47.53641171826598]
  This paper comprehensively reviews the deep learning-based methods for image and video inpainting. We sort existing methods into different categories from the perspective of their high-level inpainting pipeline. We present evaluation metrics for low-level pixel and high-level perceptional similarity, conduct a performance evaluation, and discuss the strengths and weaknesses of representative inpainting methods.
  arXiv  Detail & Related papers  (2024-01-07T05:50:12Z)
• Synergy of Machine and Deep Learning Models for Multi-Painter Recognition [0.0]
  We introduce a new large dataset for painting recognition task including 62 artists achieving good results. RegNet performs better in exporting features, while SVM makes the best classification of images based on the painter with a performance of up to 85%.
  arXiv  Detail & Related papers  (2023-04-28T11:34:53Z)
• A domain adaptive deep learning solution for scanpath prediction of paintings [66.46953851227454]
  This paper focuses on the eye-movement analysis of viewers during the visual experience of a certain number of paintings. We introduce a new approach to predicting human visual attention, which impacts several cognitive functions for humans. The proposed new architecture ingests images and returns scanpaths, a sequence of points featuring a high likelihood of catching viewers' attention.
  arXiv  Detail & Related papers  (2022-09-22T22:27:08Z)
• Compositional Scene Representation Learning via Reconstruction: A Survey [48.33349317481124]
  Compositional scene representation learning is a task that enables such abilities. Deep neural networks have been proven to be advantageous in representation learning. Learning via reconstruction is advantageous because it may utilize massive unlabeled data and avoid costly and laborious data annotation.
  arXiv  Detail & Related papers  (2022-02-15T02:14:05Z)
• Neural Fields in Visual Computing and Beyond [54.950885364735804]
  Recent advances in machine learning have created increasing interest in solving visual computing problems using coordinate-based neural networks. neural fields have seen successful application in the synthesis of 3D shapes and image, animation of human bodies, 3D reconstruction, and pose estimation. This report provides context, mathematical grounding, and an extensive review of literature on neural fields.
  arXiv  Detail & Related papers  (2021-11-22T18:57:51Z)
• Image Quality Assessment in the Modern Age [53.19271326110551]
  This tutorial provides the audience with the basic theories, methodologies, and current progresses of image quality assessment (IQA) We will first revisit several subjective quality assessment methodologies, with emphasis on how to properly select visual stimuli. Both hand-engineered and (deep) learning-based methods will be covered.
  arXiv  Detail & Related papers  (2021-10-19T02:38:46Z)
• Video Summarization Using Deep Neural Networks: A Survey [72.98424352264904]
  Video summarization technologies aim to create a concise and complete synopsis by selecting the most informative parts of the video content. This work focuses on the recent advances in the area and provides a comprehensive survey of the existing deep-learning-based methods for generic video summarization.
  arXiv  Detail & Related papers  (2021-01-15T11:41:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.