Multi-Scale Hourglass Hierarchical Fusion Network for Single Image Deraining

• URL: http://arxiv.org/abs/2104.12100v1
• Date: Sun, 25 Apr 2021 08:27:01 GMT
• Title: Multi-Scale Hourglass Hierarchical Fusion Network for Single Image Deraining
• Authors: Xiang Chen, Yufeng Huang, Lei Xu
• Abstract summary: Rain streaks bring serious blurring and visual quality degradation, which often vary in size, direction and density. Current CNN-based methods achieve encouraging performance, while are limited to depict rain characteristics and recover image details in the poor visibility environment. We present a Multi-scale Hourglass Hierarchical Fusion Network (MH2F-Net) in end-to-end manner, to exactly captures rain streak features with multi-scale extraction, hierarchical distillation and information aggregation.
• Score: 8.964751500091005
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Rain streaks bring serious blurring and visual quality degradation, which often vary in size, direction and density. Current CNN-based methods achieve encouraging performance, while are limited to depict rain characteristics and recover image details in the poor visibility environment. To address these issues, we present a Multi-scale Hourglass Hierarchical Fusion Network (MH2F-Net) in end-to-end manner, to exactly captures rain streak features with multi-scale extraction, hierarchical distillation and information aggregation. For better extracting the features, a novel Multi-scale Hourglass Extraction Block (MHEB) is proposed to get local and global features across different scales through down- and up-sample process. Besides, a Hierarchical Attentive Distillation Block (HADB) then employs the dual attention feature responses to adaptively recalibrate the hierarchical features and eliminate the redundant ones. Further, we introduce a Residual Projected Feature Fusion (RPFF) strategy to progressively discriminate feature learning and aggregate different features instead of directly concatenating or adding. Extensive experiments on both synthetic and real rainy datasets demonstrate the effectiveness of the designed MH2F-Net by comparing with recent state-of-the-art deraining algorithms. Our source code will be available on the GitHub: https://github.com/cxtalk/MH2F-Net.

Related papers

• Retain, Blend, and Exchange: A Quality-aware Spatial-Stereo Fusion Approach for Event Stream Recognition [57.74076383449153]
  We propose a novel dual-stream framework for event stream-based pattern recognition via differentiated fusion, termed EFV++. It models two common event representations simultaneously, i.e., event images and event voxels. We achieve new state-of-the-art performance on the Bullying10k dataset, i.e., $90.51%$, which exceeds the second place by $+2.21%$.
  arXiv  Detail & Related papers  (2024-06-27T02:32:46Z)
• Bidirectional Multi-Scale Implicit Neural Representations for Image Deraining [47.15857899099733]
  We develop an end-to-end multi-scale Transformer to facilitate high-quality image reconstruction. We incorporate intra-scale implicit neural representations based on pixel coordinates with the degraded inputs in a closed-loop design. Our approach, named as NeRD-Rain, performs favorably against the state-of-the-art ones on both synthetic and real-world benchmark datasets.
  arXiv  Detail & Related papers  (2024-04-02T01:18:16Z)
• Dynamic Association Learning of Self-Attention and Convolution in Image Restoration [56.49098856632478]
  CNNs and Self attention have achieved great success in multimedia applications for dynamic association learning of self-attention and convolution in image restoration. This paper proposes an association learning method to utilize the advantages and suppress their shortcomings, so as to achieve high-quality and efficient inpainting.
  arXiv  Detail & Related papers  (2023-11-09T05:11:24Z)
• Mutual-Guided Dynamic Network for Image Fusion [51.615598671899335]
  We propose a novel mutual-guided dynamic network (MGDN) for image fusion, which allows for effective information utilization across different locations and inputs. Experimental results on five benchmark datasets demonstrate that our proposed method outperforms existing methods on four image fusion tasks.
  arXiv  Detail & Related papers  (2023-08-24T03:50:37Z)
• Magic ELF: Image Deraining Meets Association Learning and Transformer [63.761812092934576]
  This paper aims to unify CNN and Transformer to take advantage of their learning merits for image deraining. A novel multi-input attention module (MAM) is proposed to associate rain removal and background recovery. Our proposed method (dubbed as ELF) outperforms the state-of-the-art approach (MPRNet) by 0.25 dB on average.
  arXiv  Detail & Related papers  (2022-07-21T12:50:54Z)
• Online-updated High-order Collaborative Networks for Single Image Deraining [51.22694467126883]
  Single image deraining is an important task for some downstream artificial intelligence applications such as video surveillance and self-driving systems. We propose a high-order collaborative network with multi-scale compact constraints and a bidirectional scale-content similarity mining module. Our proposed method performs favorably against eleven state-of-the-art methods on five public synthetic and one real-world dataset.
  arXiv  Detail & Related papers  (2022-02-14T09:09:08Z)
• Exploring Overcomplete Representations for Single Image Deraining using CNNs [0.0]
  Most recent methods for deraining use a deep network which captures low-level features across the initial layers and high-level features in the deeper layers. We propose using an overcomplete convolutional network architecture which gives special attention in learning local structures. We combine it with U-Net so that it does not lose out on the global structures as well while focusing more on low-level features.
  arXiv  Detail & Related papers  (2020-10-20T22:55:02Z)
• MCW-Net: Single Image Deraining with Multi-level Connections and Wide Regional Non-local Blocks [6.007222067550804]
  We present a multi-level connection and wide regional non-local block network (MCW-Net) to restore the original background textures in rainy images. MCW-Net improves performance by maximizing information utilization without additional branches. Experimental results on both synthetic and real-world rainy datasets demonstrate that the proposed model significantly outperforms existing state-of-the-art models.
  arXiv  Detail & Related papers  (2020-09-29T13:21:31Z)
• DCSFN: Deep Cross-scale Fusion Network for Single Image Rain Removal [13.794959799789703]
  Rain removal is an important but challenging computer vision task as rain streaks can severely degrade the visibility of images. Previous works mainly focused on feature extraction and processing or neural network structure. In this paper, we explore the cross-scale manner between networks and inner-scale fusion operation to solve the image rain removal task.
  arXiv  Detail & Related papers  (2020-08-03T10:34:45Z)
• Multi-Scale Progressive Fusion Network for Single Image Deraining [84.0466298828417]
  Rain streaks in the air appear in various blurring degrees and resolutions due to different distances from their positions to the camera. Similar rain patterns are visible in a rain image as well as its multi-scale (or multi-resolution) versions. In this work, we explore the multi-scale collaborative representation for rain streaks from the perspective of input image scales and hierarchical deep features.
  arXiv  Detail & Related papers  (2020-03-24T17:22:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.