RiWNet: A moving object instance segmentation Network being Robust in adverse Weather conditions

• URL: http://arxiv.org/abs/2109.01820v1
• Date: Sat, 4 Sep 2021 08:55:36 GMT
• Title: RiWNet: A moving object instance segmentation Network being Robust in adverse Weather conditions
• Authors: Chenjie Wang, Chengyuan Li, Bin Luo, Wei Wang, Jun Liu
• Abstract summary: We focus on a new possibility, that is, to improve its resilience to weather interference through the network's structural design. We propose a novel FPN structure called RiWFPN with a progressive top-down interaction and attention refinement module. We extend SOLOV2 to capture temporal information in video to learn motion information, and propose a moving object instance segmentation network with RiWFPN called RiWNet.
• Score: 13.272209740926156
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Segmenting each moving object instance in a scene is essential for many applications. But like many other computer vision tasks, this task performs well in optimal weather, but then adverse weather tends to fail. To be robust in weather conditions, the usual way is to train network in data of given weather pattern or to fuse multiple sensors. We focus on a new possibility, that is, to improve its resilience to weather interference through the network's structural design. First, we propose a novel FPN structure called RiWFPN with a progressive top-down interaction and attention refinement module. RiWFPN can directly replace other FPN structures to improve the robustness of the network in non-optimal weather conditions. Then we extend SOLOV2 to capture temporal information in video to learn motion information, and propose a moving object instance segmentation network with RiWFPN called RiWNet. Finally, in order to verify the effect of moving instance segmentation in different weather disturbances, we propose a VKTTI-moving dataset which is a moving instance segmentation dataset based on the VKTTI dataset, taking into account different weather scenes such as rain, fog, sunset, morning as well as overcast. The experiment proves how RiWFPN improves the network's resilience to adverse weather effects compared to other FPN structures. We compare RiWNet to several other state-of-the-art methods in some challenging datasets, and RiWNet shows better performance especially under adverse weather conditions.

Related papers

• MWFormer: Multi-Weather Image Restoration Using Degradation-Aware Transformers [44.600209414790854]
  Restoring images captured under adverse weather conditions is a fundamental task for many computer vision applications. We propose a multi-weather Transformer, or MWFormer, that aims to solve multiple weather-induced degradations using a single architecture. We show that MWFormer achieves significant performance improvements compared to existing state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-11-26T08:47:39Z)
• Genuine Knowledge from Practice: Diffusion Test-Time Adaptation for Video Adverse Weather Removal [53.15046196592023]
  We introduce test-time adaptation into adverse weather removal in videos. We propose the first framework that integrates test-time adaptation into the iterative diffusion reverse process.
  arXiv  Detail & Related papers  (2024-03-12T14:21:30Z)
• EGVD: Event-Guided Video Deraining [57.59935209162314]
  We propose an end-to-end learning-based network to unlock the potential of the event camera for video deraining. We build a real-world dataset consisting of rainy videos and temporally synchronized event streams.
  arXiv  Detail & Related papers  (2023-09-29T13:47:53Z)
• Video Adverse-Weather-Component Suppression Network via Weather Messenger and Adversarial Backpropagation [45.184188689391775]
  We propose the first framework for restoring videos from all adverse weather conditions by developing a video adverse-weather-component suppression network (ViWS-Net) Our ViWS-Net outperforms current state-of-the-art methods in terms of restoring videos degraded by any weather condition.
  arXiv  Detail & Related papers  (2023-09-24T17:13:55Z)
• Exploring the Application of Large-scale Pre-trained Models on Adverse Weather Removal [97.53040662243768]
  We propose a CLIP embedding module to make the network handle different weather conditions adaptively. This module integrates the sample specific weather prior extracted by CLIP image encoder together with the distribution specific information learned by a set of parameters.
  arXiv  Detail & Related papers  (2023-06-15T10:06:13Z)
• WM-MoE: Weather-aware Multi-scale Mixture-of-Experts for Blind Adverse Weather Removal [38.257012295118145]
  Mixture-of-Experts (MoE) is a popular model that adopts a learnable gate to route the input to different expert networks. We propose Weather-aware Multi-scale MoE (WM-MoE) based on Transformer for blind weather removal. Our method achieves state-of-the-art performance in blind adverse weather removal on two public datasets and our dataset.
  arXiv  Detail & Related papers  (2023-03-24T01:46:25Z)
• TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions [77.20136060506906]
  We propose TransWeather, a transformer-based end-to-end model with just a single encoder and a decoder. TransWeather achieves significant improvements across multiple test datasets over both All-in-One network. It is validated on real world test images and found to be more effective than previous methods.
  arXiv  Detail & Related papers  (2021-11-29T18:57:09Z)
• Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of Adverse Weather Conditions for 3D Object Detection [60.89616629421904]
  Lidar-based object detectors are critical parts of the 3D perception pipeline in autonomous navigation systems such as self-driving cars. They are sensitive to adverse weather conditions such as rain, snow and fog due to reduced signal-to-noise ratio (SNR) and signal-to-background ratio (SBR)
  arXiv  Detail & Related papers  (2021-07-14T21:10:47Z)
• Mutual-GAN: Towards Unsupervised Cross-Weather Adaptation with Mutual Information Constraint [32.67453558911618]
  Convolutional neural network (CNN) have proven its success for semantic segmentation, which is a core task of emerging industrial applications such as autonomous driving. In practical applications, the outdoor weather and illumination are changeable, e.g., cloudy and nighttime, which results in a significant drop of semantic segmentation accuracy of CNN only trained with daytime data. We propose a novel generative adversarial network (namely Mutual-GAN) to alleviate the accuracy decline when daytime-trained neural network is applied to videos captured under adverse weather conditions.
  arXiv  Detail & Related papers  (2021-06-30T11:44:22Z)
• Weather GAN: Multi-Domain Weather Translation Using Generative Adversarial Networks [76.64158017926381]
  A new task is proposed, namely, weather translation, which refers to transferring weather conditions of the image from one category to another. We develop a multi-domain weather translation approach based on generative adversarial networks (GAN), denoted as Weather GAN. Our approach suppresses the distortion and deformation caused by weather translation.
  arXiv  Detail & Related papers  (2021-03-09T13:51:58Z)
• Robust Semantic Segmentation in Adverse Weather Conditions by means of Fast Video-Sequence Segmentation [12.788867618508688]
  Video-segmentation approaches capture temporal information of the previous video-frames in addition to current image information. Video-segmentation approaches, which are often based on recurrent neural networks, cannot be applied in real-time applications anymore. In this work, the LSTM-ICNet is sped up by modifying the recurrent units of the network so that it becomes real-time capable again.
  arXiv  Detail & Related papers  (2020-07-01T07:29:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.