Snow Mask Guided Adaptive Residual Network for Image Snow Removal

• URL: http://arxiv.org/abs/2207.04754v1
• Date: Mon, 11 Jul 2022 10:30:46 GMT
• Title: Snow Mask Guided Adaptive Residual Network for Image Snow Removal
• Authors: Bodong Cheng, Juncheng Li, Ying Chen, Shuyi Zhang, Tieyong Zeng
• Abstract summary: Snow is an extremely common atmospheric phenomenon that will seriously affect the performance of high-level computer vision tasks. We propose a Snow Mask Guided Adaptive Residual Network (SMGARN) It consists of three parts, Mask-Net, Guidance-Fusion Network (GF-Net), and Reconstruct-Net. Our SMGARN numerically outperforms all existing snow removal methods, and the reconstructed images are clearer in visual contrast.
• Score: 21.228758052455273
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Image restoration under severe weather is a challenging task. Most of the past works focused on removing rain and haze phenomena in images. However, snow is also an extremely common atmospheric phenomenon that will seriously affect the performance of high-level computer vision tasks, such as object detection and semantic segmentation. Recently, some methods have been proposed for snow removing, and most methods deal with snow images directly as the optimization object. However, the distribution of snow location and shape is complex. Therefore, failure to detect snowflakes / snow streak effectively will affect snow removing and limit the model performance. To solve these issues, we propose a Snow Mask Guided Adaptive Residual Network (SMGARN). Specifically, SMGARN consists of three parts, Mask-Net, Guidance-Fusion Network (GF-Net), and Reconstruct-Net. Firstly, we build a Mask-Net with Self-pixel Attention (SA) and Cross-pixel Attention (CA) to capture the features of snowflakes and accurately localized the location of the snow, thus predicting an accurate snow mask. Secondly, the predicted snow mask is sent into the specially designed GF-Net to adaptively guide the model to remove snow. Finally, an efficient Reconstruct-Net is used to remove the veiling effect and correct the image to reconstruct the final snow-free image. Extensive experiments show that our SMGARN numerically outperforms all existing snow removal methods, and the reconstructed images are clearer in visual contrast. All codes will be available.

Related papers

• A deep learning approach for marine snow synthesis and removal [55.86191108738564]
  This paper proposes a novel method to reduce the marine snow interference using deep learning techniques. We first synthesize realistic marine snow samples by training a Generative Adversarial Network (GAN) model. We then train a U-Net model to perform marine snow removal as an image to image translation task.
  arXiv  Detail & Related papers  (2023-11-27T07:19:41Z)
• ScatterNeRF: Seeing Through Fog with Physically-Based Inverse Neural Rendering [83.75284107397003]
  We introduce ScatterNeRF, a neural rendering method which renders scenes and decomposes the fog-free background. We propose a disentangled representation for the scattering volume and the scene objects, and learn the scene reconstruction with physics-inspired losses. We validate our method by capturing multi-view In-the-Wild data and controlled captures in a large-scale fog chamber.
  arXiv  Detail & Related papers  (2023-05-03T13:24:06Z)
• Star-Net: Improving Single Image Desnowing Model With More Efficient Connection and Diverse Feature Interaction [0.8602553195689513]
  We propose a novel single image desnowing network called Star-Net. First, we design a Star type Skip Connection (SSC) to establish information channels for all different scale features. Second, we present a Multi-Stage Interactive Transformer (MIT) as the base module of Star-Net. Third, we propose a Degenerate Filter Module (DFM) to filter the snow particle and snow fog residual in the SSC.
  arXiv  Detail & Related papers  (2023-03-17T14:03:49Z)
• LMQFormer: A Laplace-Prior-Guided Mask Query Transformer for Lightweight Snow Removal [22.047433543495867]
  We propose a lightweight but high-efficient snow removal network called Laplace Mask Query Transformer (LMQFormer) Firstly, we present a Laplace-VQVAE to generate a coarse mask as prior knowledge of snow. Instead of using the mask in dataset, we aim at reducing both the information entropy of snow and the computational cost of recovery. Thirdly, we develop a Duplicated Mask Query Attention (DMQA) that converts the coarse mask into a specific number of queries, which constraint the attention areas of MQFormer with reduced parameters.
  arXiv  Detail & Related papers  (2022-10-10T15:44:06Z)
• SnowFormer: Scale-aware Transformer via Context Interaction for Single Image Desnowing [9.747362856056162]
  We propose a powerful architecture dubbed as SnowFormer for single image desnowing. It performs Scale-aware Feature Aggregation in the encoder to capture rich snow information of various degradations. It also uses a novel Context Interaction Transformer Block in the decoder, which conducts context interaction of local details and global information.
  arXiv  Detail & Related papers  (2022-08-20T15:01:09Z)
• MSP-Former: Multi-Scale Projection Transformer for Single Image Desnowing [6.22867695581195]
  We apply the vision transformer to the task of snow removal from a single image. We propose a parallel network architecture split along the channel, performing local feature refinement and global information modeling separately. In the experimental part, we conduct extensive experiments to demonstrate the superiority of our method.
  arXiv  Detail & Related papers  (2022-07-12T15:44:07Z)
• LiDAR Snowfall Simulation for Robust 3D Object Detection [116.10039516404743]
  We propose a physically based method to simulate the effect of snowfall on real clear-weather LiDAR point clouds. Our method samples snow particles in 2D space for each LiDAR line and uses the induced geometry to modify the measurement for each LiDAR beam. We use our simulation to generate partially synthetic snowy LiDAR data and leverage these data for training 3D object detection models that are robust to snowfall.
  arXiv  Detail & Related papers  (2022-03-28T21:48:26Z)
• TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions [77.20136060506906]
  We propose TransWeather, a transformer-based end-to-end model with just a single encoder and a decoder. TransWeather achieves significant improvements across multiple test datasets over both All-in-One network. It is validated on real world test images and found to be more effective than previous methods.
  arXiv  Detail & Related papers  (2021-11-29T18:57:09Z)
• Beyond Monocular Deraining: Parallel Stereo Deraining Network Via Semantic Prior [103.49307603952144]
  Most existing de-rain algorithms use only one single input image and aim to recover a clean image. We present a Paired Rain Removal Network (PRRNet), which exploits both stereo images and semantic information. Experiments on both monocular and the newly proposed stereo rainy datasets demonstrate that the proposed method achieves the state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-05-09T04:15:10Z)
• Deep Dense Multi-scale Network for Snow Removal Using Semantic and Geometric Priors [78.61844008368587]
  We propose a Deep Dense Multi-Scale Network (textbfDDMSNet) for snow removal by exploiting semantic and geometric priors. We incorporate the semantic and geometric maps as input and learn the semantic-aware and geometry-aware representation to remove snow.
  arXiv  Detail & Related papers  (2021-03-21T03:30:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.