F2T2-HiT: A U-Shaped FFT Transformer and Hierarchical Transformer for Reflection Removal

• URL: http://arxiv.org/abs/2506.05489v1
• Date: Thu, 05 Jun 2025 18:12:36 GMT
• Title: F2T2-HiT: A U-Shaped FFT Transformer and Hierarchical Transformer for Reflection Removal
• Authors: Jie Cai, Kangning Yang, Ling Ouyang, Lan Fu, Jiaming Ding, Huiming Sun, Chiu Man Ho, Zibo Meng,
• Abstract summary: Single Image Reflection Removal (SIRR) technique plays a crucial role in image processing by eliminating unwanted reflections from the background.<n>These reflections, often caused by photographs taken through glass surfaces, can significantly degrade image quality.<n>This paper introduces a U-shaped Fast Fourier Transform Transformer and Hierarchical Transformer architecture, an innovative Transformer-based design for SIRR.
• Score: 16.539156634006236
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Single Image Reflection Removal (SIRR) technique plays a crucial role in image processing by eliminating unwanted reflections from the background. These reflections, often caused by photographs taken through glass surfaces, can significantly degrade image quality. SIRR remains a challenging problem due to the complex and varied reflections encountered in real-world scenarios. These reflections vary significantly in intensity, shapes, light sources, sizes, and coverage areas across the image, posing challenges for most existing methods to effectively handle all cases. To address these challenges, this paper introduces a U-shaped Fast Fourier Transform Transformer and Hierarchical Transformer (F2T2-HiT) architecture, an innovative Transformer-based design for SIRR. Our approach uniquely combines Fast Fourier Transform (FFT) Transformer blocks and Hierarchical Transformer blocks within a UNet framework. The FFT Transformer blocks leverage the global frequency domain information to effectively capture and separate reflection patterns, while the Hierarchical Transformer blocks utilize multi-scale feature extraction to handle reflections of varying sizes and complexities. Extensive experiments conducted on three publicly available testing datasets demonstrate state-of-the-art performance, validating the effectiveness of our approach.

Related papers

• FiTv2: Scalable and Improved Flexible Vision Transformer for Diffusion Model [76.84519526283083]
  We present the textbfFlexible Vision Transformer (FiT), a transformer architecture specifically designed for generating images with textitunrestricted resolutions and aspect ratios FiTv2 exhibits $2times$ convergence speed of FiT, when incorporating advanced training-free extrapolation techniques. Comprehensive experiments demonstrate the exceptional performance of FiTv2 across a broad range of resolutions.
  arXiv  Detail & Related papers  (2024-10-17T15:51:49Z)
• Effective Diffusion Transformer Architecture for Image Super-Resolution [63.254644431016345]
  We design an effective diffusion transformer for image super-resolution (DiT-SR) In practice, DiT-SR leverages an overall U-shaped architecture, and adopts a uniform isotropic design for all the transformer blocks. We analyze the limitation of the widely used AdaLN, and present a frequency-adaptive time-step conditioning module.
  arXiv  Detail & Related papers  (2024-09-29T07:14:16Z)
• FiT: Flexible Vision Transformer for Diffusion Model [81.85667773832279]
  We present a transformer architecture specifically designed for generating images with unrestricted resolutions and aspect ratios. Unlike traditional methods that perceive images as static-resolution grids, FiT conceptualizes images as sequences of dynamically-sized tokens. Comprehensive experiments demonstrate the exceptional performance of FiT across a broad range of resolutions.
  arXiv  Detail & Related papers  (2024-02-19T18:59:07Z)
• PFT-SSR: Parallax Fusion Transformer for Stereo Image Super-Resolution [22.251884516076096]
  We propose a novel Transformer-based parallax fusion module called Parallax Fusion Transformer (PFT) PFT employs a Cross-view Fusion Transformer (CVFT) to utilize cross-view information and an Intra-view Refinement Transformer (IVRT) for intra-view feature refinement. Experiments and ablation studies show that PFT-SSR achieves competitive results and outperforms most SOTA methods.
  arXiv  Detail & Related papers  (2023-03-24T05:04:52Z)
• Contextual Learning in Fourier Complex Field for VHR Remote Sensing Images [64.84260544255477]
  transformer-based models demonstrated outstanding potential for learning high-order contextual relationships from natural images with general resolution (224x224 pixels) We propose a complex self-attention (CSA) mechanism to model the high-order contextual information with less than half computations of naive SA. By stacking various layers of CSA blocks, we propose the Fourier Complex Transformer (FCT) model to learn global contextual information from VHR aerial images.
  arXiv  Detail & Related papers  (2022-10-28T08:13:33Z)
• Towards Lightweight Transformer via Group-wise Transformation for Vision-and-Language Tasks [126.33843752332139]
  We introduce Group-wise Transformation towards a universal yet lightweight Transformer for vision-and-language tasks, termed as LW-Transformer. We apply LW-Transformer to a set of Transformer-based networks, and quantitatively measure them on three vision-and-language tasks and six benchmark datasets. Experimental results show that while saving a large number of parameters and computations, LW-Transformer achieves very competitive performance against the original Transformer networks for vision-and-language tasks.
  arXiv  Detail & Related papers  (2022-04-16T11:30:26Z)
• U2-Former: A Nested U-shaped Transformer for Image Restoration [30.187257111046556]
  We present a deep and effective Transformer-based network for image restoration, termed as U2-Former. It is able to employ Transformer as the core operation to perform image restoration in a deep encoding and decoding space.
  arXiv  Detail & Related papers  (2021-12-04T08:37:04Z)
• PPT Fusion: Pyramid Patch Transformerfor a Case Study in Image Fusion [37.993611194758195]
  We propose a Patch PyramidTransformer(PPT) to address the issues of extracting semantic information from an image. The experimental results demonstrate its superior performance against the state-of-the-art fusion approaches.
  arXiv  Detail & Related papers  (2021-07-29T13:57:45Z)
• Diverse Image Inpainting with Bidirectional and Autoregressive Transformers [55.21000775547243]
  We propose BAT-Fill, an image inpainting framework with a novel bidirectional autoregressive transformer (BAT) BAT-Fill inherits the merits of transformers and CNNs in a two-stage manner, which allows to generate high-resolution contents without being constrained by the quadratic complexity of attention in transformers.
  arXiv  Detail & Related papers  (2021-04-26T03:52:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.