Dual-Path Multi-Scale Transformer for High-Quality Image Deraining

• URL: http://arxiv.org/abs/2405.18124v1
• Date: Tue, 28 May 2024 12:31:23 GMT
• Title: Dual-Path Multi-Scale Transformer for High-Quality Image Deraining
• Authors: Huiling Zhou, Xianhao Wu, Hongming Chen,
• Abstract summary: We propose a dual-path multi-scale Transformer (DPMformer) for high-quality image reconstruction. This method consists of a backbone path and two branch paths from two different multi-scale approaches. Our method achieves promising performance compared to other state-of-the-art methods.
• Score: 1.7104836047593197
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Despite the superiority of convolutional neural networks (CNNs) and Transformers in single-image rain removal, current multi-scale models still face significant challenges due to their reliance on single-scale feature pyramid patterns. In this paper, we propose an effective rain removal method, the dual-path multi-scale Transformer (DPMformer) for high-quality image reconstruction by leveraging rich multi-scale information. This method consists of a backbone path and two branch paths from two different multi-scale approaches. Specifically, one path adopts the coarse-to-fine strategy, progressively downsampling the image to 1/2 and 1/4 scales, which helps capture fine-scale potential rain information fusion. Simultaneously, we employ the multi-patch stacked model (non-overlapping blocks of size 2 and 4) to enrich the feature information of the deep network in the other path. To learn a richer blend of features, the backbone path fully utilizes the multi-scale information to achieve high-quality rain removal image reconstruction. Extensive experiments on benchmark datasets demonstrate that our method achieves promising performance compared to other state-of-the-art methods.

Related papers

• Learning Efficient and Effective Trajectories for Differential Equation-based Image Restoration [59.744840744491945]
  We reformulate the trajectory optimization of this kind of method, focusing on enhancing both reconstruction quality and efficiency. We propose cost-aware trajectory distillation to streamline complex paths into several manageable steps with adaptable sizes. Experiments showcase the significant superiority of the proposed method, achieving a maximum PSNR improvement of 2.1 dB over state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-10-07T07:46:08Z)
• Lightweight single-image super-resolution network based on dual paths [0.552480439325792]
  Single image super-resolution(SISR) algorithms under deep learning currently have two main models, one based on convolutional neural networks and the other based on Transformer. This paper proposes a new lightweight multi-scale feature fusion network model based on two-way complementary convolutional and Transformer.
  arXiv  Detail & Related papers  (2024-09-10T15:31:37Z)
• Bidirectional Multi-Scale Implicit Neural Representations for Image Deraining [47.15857899099733]
  We develop an end-to-end multi-scale Transformer to facilitate high-quality image reconstruction. We incorporate intra-scale implicit neural representations based on pixel coordinates with the degraded inputs in a closed-loop design. Our approach, named as NeRD-Rain, performs favorably against the state-of-the-art ones on both synthetic and real-world benchmark datasets.
  arXiv  Detail & Related papers  (2024-04-02T01:18:16Z)
• Online-updated High-order Collaborative Networks for Single Image Deraining [51.22694467126883]
  Single image deraining is an important task for some downstream artificial intelligence applications such as video surveillance and self-driving systems. We propose a high-order collaborative network with multi-scale compact constraints and a bidirectional scale-content similarity mining module. Our proposed method performs favorably against eleven state-of-the-art methods on five public synthetic and one real-world dataset.
  arXiv  Detail & Related papers  (2022-02-14T09:09:08Z)
• Multi-Scale Hourglass Hierarchical Fusion Network for Single Image Deraining [8.964751500091005]
  Rain streaks bring serious blurring and visual quality degradation, which often vary in size, direction and density. Current CNN-based methods achieve encouraging performance, while are limited to depict rain characteristics and recover image details in the poor visibility environment. We present a Multi-scale Hourglass Hierarchical Fusion Network (MH2F-Net) in end-to-end manner, to exactly captures rain streak features with multi-scale extraction, hierarchical distillation and information aggregation.
  arXiv  Detail & Related papers  (2021-04-25T08:27:01Z)
• M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection [74.19291916812921]
  forged images generated by Deepfake techniques pose a serious threat to the trustworthiness of digital information. In this paper, we aim to capture the subtle manipulation artifacts at different scales for Deepfake detection. We introduce a high-quality Deepfake dataset, SR-DF, which consists of 4,000 DeepFake videos generated by state-of-the-art face swapping and facial reenactment methods.
  arXiv  Detail & Related papers  (2021-04-20T05:43:44Z)
• Adaptive Context-Aware Multi-Modal Network for Depth Completion [107.15344488719322]
  We propose to adopt the graph propagation to capture the observed spatial contexts. We then apply the attention mechanism on the propagation, which encourages the network to model the contextual information adaptively. Finally, we introduce the symmetric gated fusion strategy to exploit the extracted multi-modal features effectively. Our model, named Adaptive Context-Aware Multi-Modal Network (ACMNet), achieves the state-of-the-art performance on two benchmarks.
  arXiv  Detail & Related papers  (2020-08-25T06:00:06Z)
• Multi-Scale Progressive Fusion Network for Single Image Deraining [84.0466298828417]
  Rain streaks in the air appear in various blurring degrees and resolutions due to different distances from their positions to the camera. Similar rain patterns are visible in a rain image as well as its multi-scale (or multi-resolution) versions. In this work, we explore the multi-scale collaborative representation for rain streaks from the perspective of input image scales and hierarchical deep features.
  arXiv  Detail & Related papers  (2020-03-24T17:22:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.