Star-Net: Improving Single Image Desnowing Model With More Efficient Connection and Diverse Feature Interaction

• URL: http://arxiv.org/abs/2303.09988v1
• Date: Fri, 17 Mar 2023 14:03:49 GMT
• Title: Star-Net: Improving Single Image Desnowing Model With More Efficient Connection and Diverse Feature Interaction
• Authors: Jiawei Mao, Yuanqi Chang, Xuesong Yin, Binling Nie
• Abstract summary: We propose a novel single image desnowing network called Star-Net. First, we design a Star type Skip Connection (SSC) to establish information channels for all different scale features. Second, we present a Multi-Stage Interactive Transformer (MIT) as the base module of Star-Net. Third, we propose a Degenerate Filter Module (DFM) to filter the snow particle and snow fog residual in the SSC.
• Score: 0.8602553195689513
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Compared to other severe weather image restoration tasks, single image desnowing is a more challenging task. This is mainly due to the diversity and irregularity of snow shape, which makes it extremely difficult to restore images in snowy scenes. Moreover, snow particles also have a veiling effect similar to haze or mist. Although current works can effectively remove snow particles with various shapes, they also bring distortion to the restored image. To address these issues, we propose a novel single image desnowing network called Star-Net. First, we design a Star type Skip Connection (SSC) to establish information channels for all different scale features, which can deal with the complex shape of snow particles.Second, we present a Multi-Stage Interactive Transformer (MIT) as the base module of Star-Net, which is designed to better understand snow particle shapes and to address image distortion by explicitly modeling a variety of important image recovery features. Finally, we propose a Degenerate Filter Module (DFM) to filter the snow particle and snow fog residual in the SSC on the spatial and channel domains. Extensive experiments show that our Star-Net achieves state-of-the-art snow removal performances on three standard snow removal datasets and retains the original sharpness of the images.

Related papers

• Few-shot point cloud reconstruction and denoising via learned Guassian splats renderings and fine-tuned diffusion features [52.62053703535824]
  We propose a method to reconstruct point clouds from few images and to denoise point clouds from their rendering. To improve reconstruction in constraint settings, we regularize the training of a differentiable with hybrid surface and appearance. We demonstrate how these learned filters can be used to remove point cloud noise coming without 3D supervision.
  arXiv  Detail & Related papers  (2024-04-01T13:38:16Z)
• Curricular Contrastive Regularization for Physics-aware Single Image Dehazing [56.392696439577165]
  We propose a novel curricular contrastive regularization targeted at a consensual contrastive space as opposed to a non-consensual one. Our negatives, which provide better lower-bound constraints, can be assembled from 1) the hazy image, and 2) corresponding restorations by other existing methods. With the unit, as well as curricular contrastive regularization, we establish our dehazing network, named C2PNet.
  arXiv  Detail & Related papers  (2023-03-24T18:18:25Z)
• NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusion [107.67277084886929]
  Novel view synthesis from a single image requires inferring occluded regions of objects and scenes whilst simultaneously maintaining semantic and physical consistency with the input. We propose NerfDiff, which addresses this issue by distilling the knowledge of a 3D-aware conditional diffusion model (CDM) into NeRF through synthesizing and refining a set of virtual views at test time. We further propose a novel NeRF-guided distillation algorithm that simultaneously generates 3D consistent virtual views from the CDM samples, and finetunes the NeRF based on the improved virtual views.
  arXiv  Detail & Related papers  (2023-02-20T17:12:00Z)
• See Blue Sky: Deep Image Dehaze Using Paired and Unpaired Training Images [73.23687409870656]
  We propose a cycle generative adversarial network to construct a novel end-to-end image dehaze model. We adopt outdoor image datasets to train our model, which includes a set of real-world unpaired image dataset and a set of paired image dataset. Based on the cycle structure, our model adds four different kinds of loss function to constrain the effect including adversarial loss, cycle consistency loss, photorealism loss and paired L1 loss.
  arXiv  Detail & Related papers  (2022-10-14T07:45:33Z)
• SnowFormer: Scale-aware Transformer via Context Interaction for Single Image Desnowing [9.747362856056162]
  We propose a powerful architecture dubbed as SnowFormer for single image desnowing. It performs Scale-aware Feature Aggregation in the encoder to capture rich snow information of various degradations. It also uses a novel Context Interaction Transformer Block in the decoder, which conducts context interaction of local details and global information.
  arXiv  Detail & Related papers  (2022-08-20T15:01:09Z)
• MSP-Former: Multi-Scale Projection Transformer for Single Image Desnowing [6.22867695581195]
  We apply the vision transformer to the task of snow removal from a single image. We propose a parallel network architecture split along the channel, performing local feature refinement and global information modeling separately. In the experimental part, we conduct extensive experiments to demonstrate the superiority of our method.
  arXiv  Detail & Related papers  (2022-07-12T15:44:07Z)
• Towards Real-time High-Definition Image Snow Removal: Efficient Pyramid Network with Asymmetrical Encoder-decoder Architecture [6.682410871522934]
  We develop a novel Efficient Pyramid Network with asymmetrical encoder-decoder architecture for real-time HD image desnowing. Our approach achieves a better complexity-performance trade-off and effectively handles the processing difficulties of HD and Ultra-HD images.
  arXiv  Detail & Related papers  (2022-07-12T15:18:41Z)
• Snow Mask Guided Adaptive Residual Network for Image Snow Removal [21.228758052455273]
  Snow is an extremely common atmospheric phenomenon that will seriously affect the performance of high-level computer vision tasks. We propose a Snow Mask Guided Adaptive Residual Network (SMGARN) It consists of three parts, Mask-Net, Guidance-Fusion Network (GF-Net), and Reconstruct-Net. Our SMGARN numerically outperforms all existing snow removal methods, and the reconstructed images are clearer in visual contrast.
  arXiv  Detail & Related papers  (2022-07-11T10:30:46Z)
• Deep Dense Multi-scale Network for Snow Removal Using Semantic and Geometric Priors [78.61844008368587]
  We propose a Deep Dense Multi-Scale Network (textbfDDMSNet) for snow removal by exploiting semantic and geometric priors. We incorporate the semantic and geometric maps as input and learn the semantic-aware and geometry-aware representation to remove snow.
  arXiv  Detail & Related papers  (2021-03-21T03:30:30Z)
• NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections [47.9463405062868]
  We present a learning-based method for synthesizing novel views of complex scenes using only unstructured collections of in-the-wild photographs. We build on Neural Radiance Fields (NeRF), which uses the weights of a multilayer perceptron to model the density and color of a scene as a function of 3D coordinates. We introduce a series of extensions to NeRF to address these issues, thereby enabling accurate reconstructions from unstructured image collections taken from the internet.
  arXiv  Detail & Related papers  (2020-08-05T17:51:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.