Restoration of User Videos Shared on Social Media

• URL: http://arxiv.org/abs/2208.08597v1
• Date: Thu, 18 Aug 2022 02:28:43 GMT
• Title: Restoration of User Videos Shared on Social Media
• Authors: Hongming Luo, Fei Zhou, Kin-man Lam, and Guoping Qiu
• Abstract summary: User videos shared on social media platforms usually suffer from degradations caused by unknown proprietary processing procedures. This paper presents a new general video restoration framework for the restoration of user videos shared on social media platforms. In contrast to most deep learning-based video restoration methods that perform end-to-end mapping, our new method, Video restOration through adapTive dEgradation Sensing (VOTES), introduces the concept of a degradation feature map (DFM) to explicitly guide the video restoration process.
• Score: 27.16457737969977
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: User videos shared on social media platforms usually suffer from degradations caused by unknown proprietary processing procedures, which means that their visual quality is poorer than that of the originals. This paper presents a new general video restoration framework for the restoration of user videos shared on social media platforms. In contrast to most deep learning-based video restoration methods that perform end-to-end mapping, where feature extraction is mostly treated as a black box, in the sense that what role a feature plays is often unknown, our new method, termed Video restOration through adapTive dEgradation Sensing (VOTES), introduces the concept of a degradation feature map (DFM) to explicitly guide the video restoration process. Specifically, for each video frame, we first adaptively estimate its DFM to extract features representing the difficulty of restoring its different regions. We then feed the DFM to a convolutional neural network (CNN) to compute hierarchical degradation features to modulate an end-to-end video restoration backbone network, such that more attention is paid explicitly to potentially more difficult to restore areas, which in turn leads to enhanced restoration performance. We will explain the design rationale of the VOTES framework and present extensive experimental results to show that the new VOTES method outperforms various state-of-the-art techniques both quantitatively and qualitatively. In addition, we contribute a large scale real-world database of user videos shared on different social media platforms. Codes and datasets are available at https://github.com/luohongming/VOTES.git

Related papers

• Learning Truncated Causal History Model for Video Restoration [14.381907888022615]
  TURTLE learns the truncated causal history model for efficient and high-performing video restoration. We report new state-of-the-art results on a multitude of video restoration benchmark tasks.
  arXiv  Detail & Related papers  (2024-10-04T21:31:02Z)
• DiffIR2VR-Zero: Zero-Shot Video Restoration with Diffusion-based Image Restoration Models [9.145545884814327]
  This paper introduces a method for zero-shot video restoration using pre-trained image restoration diffusion models. We show that our method achieves top performance in zero-shot video restoration. Our technique works with any 2D restoration diffusion model, offering a versatile and powerful tool for video enhancement tasks without extensive retraining.
  arXiv  Detail & Related papers  (2024-07-01T17:59:12Z)
• Dual-Stream Knowledge-Preserving Hashing for Unsupervised Video Retrieval [67.52910255064762]
  We design a simple dual-stream structure, including a temporal layer and a hash layer. We first design a simple dual-stream structure, including a temporal layer and a hash layer. With the help of semantic similarity knowledge obtained from self-supervision, the hash layer learns to capture information for semantic retrieval. In this way, the model naturally preserves the disentangled semantics into binary codes.
  arXiv  Detail & Related papers  (2023-10-12T03:21:12Z)
• Video Infringement Detection via Feature Disentanglement and Mutual Information Maximization [51.206398602941405]
  We propose to disentangle an original high-dimensional feature into multiple sub-features. On top of the disentangled sub-features, we learn an auxiliary feature to enhance the sub-features. Our method achieves 90.1% TOP-100 mAP on the large-scale SVD dataset and also sets the new state-of-the-art on the VCSL benchmark dataset.
  arXiv  Detail & Related papers  (2023-09-13T10:53:12Z)
• TVTSv2: Learning Out-of-the-box Spatiotemporal Visual Representations at Scale [59.01246141215051]
  We analyze the factor that leads to degradation from the perspective of language supervision. We propose a tunable-free pre-training strategy to retain the generalization ability of the text encoder. We produce a series of models, dubbed TVTSv2, with up to one billion parameters.
  arXiv  Detail & Related papers  (2023-05-23T15:44:56Z)
• Video Event Restoration Based on Keyframes for Video Anomaly Detection [9.18057851239942]
  Existing deep neural network based anomaly detection (VAD) methods mostly follow the route of frame reconstruction or frame prediction. We introduce a brand-new VAD paradigm to break through these limitations. We propose a novel U-shaped Swin Transformer Network with Dual Skip Connections (USTN-DSC) for video event restoration.
  arXiv  Detail & Related papers  (2023-04-11T10:13:19Z)
• On the Generalization of BasicVSR++ to Video Deblurring and Denoising [98.99165593274304]
  We extend BasicVSR++ to a generic framework for video restoration tasks. In tasks where inputs and outputs possess identical spatial size, the input resolution is reduced by strided convolutions to maintain efficiency. With only minimal changes from BasicVSR++, the proposed framework achieves compelling performance with great efficiency in various video restoration tasks.
  arXiv  Detail & Related papers  (2022-04-11T17:59:56Z)
• Bringing Old Films Back to Life [33.78936333249432]
  We present a learning-based framework, recurrent transformer network (RTN), to restore heavily degraded old films. Our method is based on the hidden knowledge learned from adjacent frames that contain abundant information about the occlusion.
  arXiv  Detail & Related papers  (2022-03-31T17:59:59Z)
• BasicVSR++: Improving Video Super-Resolution with Enhanced Propagation and Alignment [90.81396836308085]
  We show that by empowering recurrent framework with enhanced propagation and alignment, one can exploit video information more effectively. Our model BasicVSR++ surpasses BasicVSR by 0.82 dB in PSNR with similar number of parameters. BasicVSR++ generalizes well to other video restoration tasks such as compressed video enhancement.
  arXiv  Detail & Related papers  (2021-04-27T17:58:31Z)
• Learning the Loss Functions in a Discriminative Space for Video Restoration [48.104095018697556]
  We propose a new framework for building effective loss functions by learning a discriminative space specific to a video restoration task. Our framework is similar to GANs in that we iteratively train two networks - a generator and a loss network. Experiments on video superresolution and deblurring show that our method generates visually more pleasing videos.
  arXiv  Detail & Related papers  (2020-03-20T06:58:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.