Structural Prior Guided Generative Adversarial Transformers for Low-Light Image Enhancement

• URL: http://arxiv.org/abs/2207.07828v2
• Date: Tue, 19 Jul 2022 04:00:36 GMT
• Title: Structural Prior Guided Generative Adversarial Transformers for Low-Light Image Enhancement
• Authors: Cong Wang and Jinshan Pan and Xiao-Ming Wu
• Abstract summary: We propose an effective Structural Prior guided Generative Adversarial Transformer (SPGAT) to solve low-light image enhancement. The generator is based on a U-shaped Transformer which is used to explore non-local information for better clear image restoration. To generate more realistic images, we develop a new structural prior guided adversarial learning method by building the skip connections between the generator and discriminators.
• Score: 51.22694467126883
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We propose an effective Structural Prior guided Generative Adversarial Transformer (SPGAT) to solve low-light image enhancement. Our SPGAT mainly contains a generator with two discriminators and a structural prior estimator (SPE). The generator is based on a U-shaped Transformer which is used to explore non-local information for better clear image restoration. The SPE is used to explore useful structures from images to guide the generator for better structural detail estimation. To generate more realistic images, we develop a new structural prior guided adversarial learning method by building the skip connections between the generator and discriminators so that the discriminators can better discriminate between real and fake features. Finally, we propose a parallel windows-based Swin Transformer block to aggregate different level hierarchical features for high-quality image restoration. Experimental results demonstrate that the proposed SPGAT performs favorably against recent state-of-the-art methods on both synthetic and real-world datasets.

Related papers

• Exploring Structure-aware Transformer over Interaction Proposals for Human-Object Interaction Detection [119.93025368028083]
  We design a novel Transformer-style Human-Object Interaction (HOI) detector, i.e., Structure-aware Transformer over Interaction Proposals (STIP) STIP decomposes the process of HOI set prediction into two subsequent phases, i.e., an interaction proposal generation is first performed, and then followed by transforming the non-parametric interaction proposals into HOI predictions via a structure-aware Transformer. The structure-aware Transformer upgrades vanilla Transformer by encoding additionally the holistically semantic structure among interaction proposals as well as the locally spatial structure of human/object within each interaction proposal, so as to strengthen HOI
  arXiv  Detail & Related papers  (2022-06-13T16:21:08Z)
• RePre: Improving Self-Supervised Vision Transformer with Reconstructive Pre-training [80.44284270879028]
  This paper incorporates local feature learning into self-supervised vision transformers via Reconstructive Pre-training (RePre) Our RePre extends contrastive frameworks by adding a branch for reconstructing raw image pixels in parallel with the existing contrastive objective.
  arXiv  Detail & Related papers  (2022-01-18T10:24:58Z)
• CSformer: Bridging Convolution and Transformer for Compressive Sensing [65.22377493627687]
  This paper proposes a hybrid framework that integrates the advantages of leveraging detailed spatial information from CNN and the global context provided by transformer for enhanced representation learning. The proposed approach is an end-to-end compressive image sensing method, composed of adaptive sampling and recovery. The experimental results demonstrate the effectiveness of the dedicated transformer-based architecture for compressive sensing.
  arXiv  Detail & Related papers  (2021-12-31T04:37:11Z)
• The Nuts and Bolts of Adopting Transformer in GANs [124.30856952272913]
  We investigate the properties of Transformer in the generative adversarial network (GAN) framework for high-fidelity image synthesis. Our study leads to a new alternative design of Transformers in GAN, a convolutional neural network (CNN)-free generator termed as STrans-G.
  arXiv  Detail & Related papers  (2021-10-25T17:01:29Z)
• PPT Fusion: Pyramid Patch Transformerfor a Case Study in Image Fusion [37.993611194758195]
  We propose a Patch PyramidTransformer(PPT) to address the issues of extracting semantic information from an image. The experimental results demonstrate its superior performance against the state-of-the-art fusion approaches.
  arXiv  Detail & Related papers  (2021-07-29T13:57:45Z)
• Augmented Shortcuts for Vision Transformers [49.70151144700589]
  We study the relationship between shortcuts and feature diversity in vision transformer models. We present an augmented shortcut scheme, which inserts additional paths with learnable parameters in parallel on the original shortcuts. Experiments conducted on benchmark datasets demonstrate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2021-06-30T09:48:30Z)
• Combining Transformer Generators with Convolutional Discriminators [9.83490307808789]
  Recently proposed TransGAN is the first GAN using only transformer-based architectures. TransGAN requires data augmentation, an auxiliary super-resolution task during training, and a masking prior to guide the self-attention mechanism. We evaluate our approach by conducting a benchmark of well-known CNN discriminators, ablate the size of the transformer-based generator, and show that combining both architectural elements into a hybrid model leads to better results.
  arXiv  Detail & Related papers  (2021-05-21T07:56:59Z)
• Training Vision Transformers for Image Retrieval [32.09708181236154]
  We adopt vision transformers for generating image descriptors and train the resulting model with a metric learning objective. Our results show consistent and significant improvements of transformers over convolution-based approaches.
  arXiv  Detail & Related papers  (2021-02-10T18:56:41Z)
• Remote sensing image fusion based on Bayesian GAN [9.852262451235472]
  We build a two-stream generator network with PAN and MS images as input, which consists of three parts: feature extraction, feature fusion and image reconstruction. We leverage Markov discriminator to enhance the ability of generator to reconstruct the fusion image, so that the result image can retain more details. Experiments on QuickBird and WorldView datasets show that the model proposed in this paper can effectively fuse PAN and MS images.
  arXiv  Detail & Related papers  (2020-09-20T16:15:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.