Improving Generative Adversarial Networks for Video Super-Resolution

• URL: http://arxiv.org/abs/2406.16359v1
• Date: Mon, 24 Jun 2024 06:57:51 GMT
• Title: Improving Generative Adversarial Networks for Video Super-Resolution
• Authors: Daniel Wen,
• Abstract summary: This research explores different ways to improve generative adversarial networks for video super-resolution tasks. We evaluate our results using Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) The integration of these methods results in an 11.97% improvement in PSNR and an 8% improvement in SSIM compared to the baseline video super-resolution generative adversarial network (GAN) model.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this research, we explore different ways to improve generative adversarial networks for video super-resolution tasks from a base single image super-resolution GAN model. Our primary objective is to identify potential techniques that enhance these models and to analyze which of these techniques yield the most significant improvements. We evaluate our results using Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM). Our findings indicate that the most effective techniques include temporal smoothing, long short-term memory (LSTM) layers, and a temporal loss function. The integration of these methods results in an 11.97% improvement in PSNR and an 8% improvement in SSIM compared to the baseline video super-resolution generative adversarial network (GAN) model. This substantial improvement suggests potential further applications to enhance current state-of-the-art models.

Related papers

• Degradation-Guided One-Step Image Super-Resolution with Diffusion Priors [75.24313405671433]
  Diffusion-based image super-resolution (SR) methods have achieved remarkable success by leveraging large pre-trained text-to-image diffusion models as priors. We introduce a novel one-step SR model, which significantly addresses the efficiency issue of diffusion-based SR methods. Unlike existing fine-tuning strategies, we designed a degradation-guided Low-Rank Adaptation (LoRA) module specifically for SR.
  arXiv  Detail & Related papers  (2024-09-25T16:15:21Z)
• DGNet: Dynamic Gradient-Guided Network for Water-Related Optics Image Enhancement [77.0360085530701]
  Underwater image enhancement (UIE) is a challenging task due to the complex degradation caused by underwater environments. Previous methods often idealize the degradation process, and neglect the impact of medium noise and object motion on the distribution of image features. Our approach utilizes predicted images to dynamically update pseudo-labels, adding a dynamic gradient to optimize the network's gradient space.
  arXiv  Detail & Related papers  (2023-12-12T06:07:21Z)
• DifAugGAN: A Practical Diffusion-style Data Augmentation for GAN-based Single Image Super-resolution [88.13972071356422]
  We propose a diffusion-style data augmentation scheme for GAN-based image super-resolution (SR) methods, known as DifAugGAN. It involves adapting the diffusion process in generative diffusion models for improving the calibration of the discriminator during training. Our DifAugGAN can be a Plug-and-Play strategy for current GAN-based SISR methods to improve the calibration of the discriminator and thus improve SR performance.
  arXiv  Detail & Related papers  (2023-11-30T12:37:53Z)
• License Plate Super-Resolution Using Diffusion Models [1.3499500088995464]
  This study leverages the cutting-edge diffusion model, which has consistently outperformed other deep learning techniques in image restoration. The method achieves a 12.55% and 37.32% improvement in Peak Signal-to-Noise Ratio (PSNR) over SwinIR and ESRGAN, respectively.
  arXiv  Detail & Related papers  (2023-09-21T22:06:23Z)
• Flow-based Visual Quality Enhancer for Super-resolution Magnetic Resonance Spectroscopic Imaging [13.408365072149795]
  We propose a flow-based enhancer network to improve the visual quality of super-resolution MRSI. Our enhancer network incorporates anatomical information from additional image modalities (MRI) and uses a learnable base distribution. Our method also allows visual quality adjustment and uncertainty estimation.
  arXiv  Detail & Related papers  (2022-07-20T20:19:44Z)
• Uncovering the Over-smoothing Challenge in Image Super-Resolution: Entropy-based Quantification and Contrastive Optimization [67.99082021804145]
  We propose an explicit solution to the COO problem, called Detail Enhanced Contrastive Loss (DECLoss) DECLoss utilizes the clustering property of contrastive learning to directly reduce the variance of the potential high-resolution distribution. We evaluate DECLoss on multiple super-resolution benchmarks and demonstrate that it improves the perceptual quality of PSNR-oriented models.
  arXiv  Detail & Related papers  (2022-01-04T08:30:09Z)
• Time-domain Speech Enhancement with Generative Adversarial Learning [53.74228907273269]
  This paper proposes a new framework called Time-domain Speech Enhancement Generative Adversarial Network (TSEGAN) TSEGAN is an extension of the generative adversarial network (GAN) in time-domain with metric evaluation to mitigate the scaling problem. In addition, we provide a new method based on objective function mapping for the theoretical analysis of the performance of Metric GAN.
  arXiv  Detail & Related papers  (2021-03-30T08:09:49Z)
• Deep Iterative Residual Convolutional Network for Single Image Super-Resolution [31.934084942626257]
  We propose a deep Iterative Super-Resolution Residual Convolutional Network (ISRResCNet) It exploits the powerful image regularization and large-scale optimization techniques by training the deep network in an iterative manner with a residual learning approach. Our method with a few trainable parameters improves the results for different scaling factors in comparison with the state-of-art methods.
  arXiv  Detail & Related papers  (2020-09-07T12:54:14Z)
• Revisiting Temporal Modeling for Video Super-resolution [47.90584361677039]
  We study and compare three temporal modeling methods (2D CNN with early fusion, 3D CNN with slow fusion and Recurrent Neural Network) for video super-resolution. We also propose a novel Recurrent Residual Network (RRN) for efficient video super-resolution, where residual learning is utilized to stabilize the training of RNN.
  arXiv  Detail & Related papers  (2020-08-13T09:09:37Z)
• iSeeBetter: Spatio-temporal video super-resolution using recurrent generative back-projection networks [0.0]
  We present iSeeBetter, a novel GAN-based structural-temporal approach to video super-resolution (VSR) iSeeBetter extracts spatial and temporal information from the current and neighboring frames using the concept of recurrent back-projection networks as its generator. Our results demonstrate that iSeeBetter offers superior VSR fidelity and surpasses state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-06-13T01:36:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.