Bilateral Network with Channel Splitting Network and Transformer for Thermal Image Super-Resolution

• URL: http://arxiv.org/abs/2206.12046v1
• Date: Fri, 24 Jun 2022 02:48:15 GMT
• Title: Bilateral Network with Channel Splitting Network and Transformer for Thermal Image Super-Resolution
• Authors: Bo Yan, Leilei Cao, Fengliang Qi and Hongbin Wang
• Abstract summary: This paper introduces the technical details of our submission to PBVS-2022 challenge designing a Bilateral Network with Channel Splitting Network and Transformer. The proposed method can achieve PSNR=33.64, SSIM=0.9263 for x4 and PSNR=21.08, SSIM=0.7803 for x2 in the PBVS-2022 challenge test dataset.
• Score: 8.754793353368255
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In recent years, the Thermal Image Super-Resolution (TISR) problem has become an attractive research topic. TISR would been used in a wide range of fields, including military, medical, agricultural and animal ecology. Due to the success of PBVS-2020 and PBVS-2021 workshop challenge, the result of TISR keeps improving and attracts more researchers to sign up for PBVS-2022 challenge. In this paper, we will introduce the technical details of our submission to PBVS-2022 challenge designing a Bilateral Network with Channel Splitting Network and Transformer(BN-CSNT) to tackle the TISR problem. Firstly, we designed a context branch based on channel splitting network with transformer to obtain sufficient context information. Secondly, we designed a spatial branch with shallow transformer to extract low level features which can preserve the spatial information. Finally, for the context branch in order to fuse the features from channel splitting network and transformer, we proposed an attention refinement module, and then features from context branch and spatial branch are fused by proposed feature fusion module. The proposed method can achieve PSNR=33.64, SSIM=0.9263 for x4 and PSNR=21.08, SSIM=0.7803 for x2 in the PBVS-2022 challenge test dataset.

Related papers

• SC-HVPPNet: Spatial and Channel Hybrid-Attention Video Post-Processing Network with CNN and Transformer [23.134971252569038]
  Convolutional Neural Network (CNN) and Transformer have attracted much attention recently for video post-processing. We propose a novel spatial and channel Hybrid-Attention Video Post-Processing Network (SC-HVPPNet) SC-HVPPNet notably boosts video restoration quality, with average savings of 5.29%, 12.42%, and 13.09% for Y, U, and V components in the VTM-11.0-NNVC RA configuration.
  arXiv  Detail & Related papers  (2024-04-23T03:35:27Z)
• Cross-Spatial Pixel Integration and Cross-Stage Feature Fusion Based Transformer Network for Remote Sensing Image Super-Resolution [13.894645293832044]
  Transformer-based models have shown competitive performance in remote sensing image super-resolution (RSISR) We propose a novel transformer architecture called Cross-Spatial Pixel Integration and Cross-Stage Feature Fusion Based Transformer Network (SPIFFNet) for RSISR. Our proposed model effectively enhances global cognition and understanding of the entire image, facilitating efficient integration of features cross-stages.
  arXiv  Detail & Related papers  (2023-07-06T13:19:06Z)
• Cross-receptive Focused Inference Network for Lightweight Image Super-Resolution [64.25751738088015]
  Transformer-based methods have shown impressive performance in single image super-resolution (SISR) tasks. Transformers that need to incorporate contextual information to extract features dynamically are neglected. We propose a lightweight Cross-receptive Focused Inference Network (CFIN) that consists of a cascade of CT Blocks mixed with CNN and Transformer.
  arXiv  Detail & Related papers  (2022-07-06T16:32:29Z)
• BSRT: Improving Burst Super-Resolution with Swin Transformer and Flow-Guided Deformable Alignment [84.82352123245488]
  This work addresses the Burst Super-Resolution (BurstSR) task using a new architecture, which requires restoring a high-quality image from a sequence of noisy, misaligned, and low-resolution RAW bursts. We propose a Burst Super-Resolution Transformer (BSRT), which can significantly improve the capability of extracting inter-frame information and reconstruction. Our BSRT wins the championship in the NTIRE2022 Burst Super-Resolution Challenge.
  arXiv  Detail & Related papers  (2022-04-18T14:23:10Z)
• P2T: Pyramid Pooling Transformer for Scene Understanding [62.41912463252468]
  We build a downstream-task-oriented transformer network, dubbed Pyramid Pooling Transformer (P2T) Plugged with our pooling-based MHSA, we build a downstream-task-oriented transformer network, dubbed Pyramid Pooling Transformer (P2T)
  arXiv  Detail & Related papers  (2021-06-22T18:28:52Z)
• Improved Transformer for High-Resolution GANs [69.42469272015481]
  We introduce two key ingredients to Transformer to address this challenge. We show in the experiments that the proposed HiT achieves state-of-the-art FID scores of 31.87 and 2.95 on unconditional ImageNet $128 times 128$ and FFHQ $256 times 256$, respectively.
  arXiv  Detail & Related papers  (2021-06-14T17:39:49Z)
• TransFuse: Fusing Transformers and CNNs for Medical Image Segmentation [9.266588373318688]
  We study the problem of improving efficiency in modeling global contexts without losing localization ability for low-level details. TransFuse, a novel two-branch architecture is proposed, which combines Transformers and CNNs in a parallel style. With TransFuse, both global dependency and low-level spatial details can be efficiently captured in a much shallower manner.
  arXiv  Detail & Related papers  (2021-02-16T08:09:45Z)
• TransGAN: Two Transformers Can Make One Strong GAN [111.07699201175919]
  We conduct the first pilot study in building a GAN textbfcompletely free of convolutions, using only pure transformer-based architectures. Our vanilla GAN architecture, dubbed textbfTransGAN, consists of a memory-friendly transformer-based generator. Our best architecture achieves highly competitive performance compared to current state-of-the-art GANs based on convolutional backbones.
  arXiv  Detail & Related papers  (2021-02-14T05:24:48Z)
• Cross-Attention in Coupled Unmixing Nets for Unsupervised Hyperspectral Super-Resolution [79.97180849505294]
  We propose a novel coupled unmixing network with a cross-attention mechanism, CUCaNet, to enhance the spatial resolution of HSI. Experiments are conducted on three widely-used HS-MS datasets in comparison with state-of-the-art HSI-SR models.
  arXiv  Detail & Related papers  (2020-07-10T08:08:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.