Multi-scale Attentive Image De-raining Networks via Neural Architecture Search

• URL: http://arxiv.org/abs/2207.00728v3
• Date: Tue, 4 Apr 2023 12:41:44 GMT
• Title: Multi-scale Attentive Image De-raining Networks via Neural Architecture Search
• Authors: Lei Cai, Yuli Fu, Wanliang Huo, Youjun Xiang, Tao Zhu, Ying Zhang, Huanqiang Zeng and Delu Zeng
• Abstract summary: We develop a high-performance multi-scale attentive neural architecture search (MANAS) framework for image deraining. The proposed method formulates a new multi-scale attention search space with multiple flexible modules that are favorite to the image de-raining task. The internal multiscale attentive architecture of the de-raining network is searched automatically through a gradient-based search algorithm.
• Score: 23.53770663034919
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-scale architectures and attention modules have shown effectiveness in many deep learning-based image de-raining methods. However, manually designing and integrating these two components into a neural network requires a bulk of labor and extensive expertise. In this article, a high-performance multi-scale attentive neural architecture search (MANAS) framework is technically developed for image deraining. The proposed method formulates a new multi-scale attention search space with multiple flexible modules that are favorite to the image de-raining task. Under the search space, multi-scale attentive cells are built, which are further used to construct a powerful image de-raining network. The internal multiscale attentive architecture of the de-raining network is searched automatically through a gradient-based search algorithm, which avoids the daunting procedure of the manual design to some extent. Moreover, in order to obtain a robust image de-raining model, a practical and effective multi-to-one training strategy is also presented to allow the de-raining network to get sufficient background information from multiple rainy images with the same background scene, and meanwhile, multiple loss functions including external loss, internal loss, architecture regularization loss, and model complexity loss are jointly optimized to achieve robust de-raining performance and controllable model complexity. Extensive experimental results on both synthetic and realistic rainy images, as well as the down-stream vision applications (i.e., objection detection and segmentation) consistently demonstrate the superiority of our proposed method. The code is publicly available at https://github.com/lcai-gz/MANAS.

Related papers

• Real-Time Image Segmentation via Hybrid Convolutional-Transformer Architecture Search [49.81353382211113]
  We address the challenge of integrating multi-head self-attention into high resolution representation CNNs efficiently. We develop a multi-target multi-branch supernet method, which fully utilizes the advantages of high-resolution features. We present a series of model via Hybrid Convolutional-Transformer Architecture Search (HyCTAS) method that searched for the best hybrid combination of light-weight convolution layers and memory-efficient self-attention layers.
  arXiv  Detail & Related papers  (2024-03-15T15:47:54Z)
• Contrastive Learning Based Recursive Dynamic Multi-Scale Network for Image Deraining [47.764883957379745]
  Rain streaks significantly decrease the visibility of captured images. Existing deep learning-based image deraining methods employ manually crafted networks and learn a straightforward projection from rainy images to clear images. We propose a contrastive learning-based image deraining method that investigates the correlation between rainy and clear images.
  arXiv  Detail & Related papers  (2023-05-29T13:51:41Z)
• Online-updated High-order Collaborative Networks for Single Image Deraining [51.22694467126883]
  Single image deraining is an important task for some downstream artificial intelligence applications such as video surveillance and self-driving systems. We propose a high-order collaborative network with multi-scale compact constraints and a bidirectional scale-content similarity mining module. Our proposed method performs favorably against eleven state-of-the-art methods on five public synthetic and one real-world dataset.
  arXiv  Detail & Related papers  (2022-02-14T09:09:08Z)
• Triple-level Model Inferred Collaborative Network Architecture for Video Deraining [43.06607185181434]
  We develop a model-guided triple-level optimization framework to deduce network architecture with cooperating optimization and auto-searching mechanism. Our model shows significant improvements in fidelity and temporal consistency over the state-of-the-art works.
  arXiv  Detail & Related papers  (2021-11-08T13:09:00Z)
• MCGKT-Net: Multi-level Context Gating Knowledge Transfer Network for Single Image Deraining [4.8609514458349095]
  This study proposes a novel MCGKT-Net for boosting deraining performance. It is a naturally multi-scale learning framework being capable of exploring multi-scale attributes of rain streaks. Experiments on three benchmark datasets manifest impressive performance compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-10-19T06:21:07Z)
• NAS-DIP: Learning Deep Image Prior with Neural Architecture Search [65.79109790446257]
  Recent work has shown that the structure of deep convolutional neural networks can be used as a structured image prior. We propose to search for neural architectures that capture stronger image priors. We search for an improved network by leveraging an existing neural architecture search algorithm.
  arXiv  Detail & Related papers  (2020-08-26T17:59:36Z)
• A Model-driven Deep Neural Network for Single Image Rain Removal [52.787356046951494]
  We propose a model-driven deep neural network for the task, with fully interpretable network structures. Based on the convolutional dictionary learning mechanism for representing rain, we propose a novel single image deraining model. All the rain kernels and operators can be automatically extracted, faithfully characterizing the features of both rain and clean background layers.
  arXiv  Detail & Related papers  (2020-05-04T09:13:25Z)
• Learning Deformable Image Registration from Optimization: Perspective, Modules, Bilevel Training and Beyond [62.730497582218284]
  We develop a new deep learning based framework to optimize a diffeomorphic model via multi-scale propagation. We conduct two groups of image registration experiments on 3D volume datasets including image-to-atlas registration on brain MRI data and image-to-image registration on liver CT data.
  arXiv  Detail & Related papers  (2020-04-30T03:23:45Z)
• Multi-Task Learning Enhanced Single Image De-Raining [9.207797392774465]
  Rain removal in images is an important task in computer vision filed and attracting attentions of more and more people. In this paper, we address a non-trivial issue of removing visual effect of rain streak from a single image. Our method combines various semantic constraint task in a proposed multi-task regression model for rain removal.
  arXiv  Detail & Related papers  (2020-03-21T16:19:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.