Semantic Segmentation under Adverse Conditions: A Weather and Nighttime-aware Synthetic Data-based Approach

• URL: http://arxiv.org/abs/2210.05626v1
• Date: Tue, 11 Oct 2022 17:14:22 GMT
• Title: Semantic Segmentation under Adverse Conditions: A Weather and Nighttime-aware Synthetic Data-based Approach
• Authors: Abdulrahman Kerim, Felipe Chamone, Washington Ramos, Leandro Soriano Marcolino, Erickson R. Nascimento, Richard Jiang
• Abstract summary: Recent semantic segmentation models perform well under standard weather conditions but struggle with adverse weather conditions and nighttime. We present a novel architecture specifically designed for using synthetic training data for domain adaptation. We propose a simple yet powerful addition to DeepLabV3+ by using weather and time-of-the-day supervisors trained with multi-task learning.
• Score: 6.482184764321084
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent semantic segmentation models perform well under standard weather conditions and sufficient illumination but struggle with adverse weather conditions and nighttime. Collecting and annotating training data under these conditions is expensive, time-consuming, error-prone, and not always practical. Usually, synthetic data is used as a feasible data source to increase the amount of training data. However, just directly using synthetic data may actually harm the model's performance under normal weather conditions while getting only small gains in adverse situations. Therefore, we present a novel architecture specifically designed for using synthetic training data for domain adaptation. We propose a simple yet powerful addition to DeepLabV3+ by using weather and time-of-the-day supervisors trained with multi-task learning, making it both weather and nighttime aware, which improves its mIoU accuracy by $14$ percentage points on the ACDC dataset while maintaining a score of $75\%$ mIoU on the Cityscapes dataset. Our code is available at https://github.com/lsmcolab/Semantic-Segmentation-under-Adverse-Conditions.

Related papers

• Exploiting Local Features and Range Images for Small Data Real-Time Point Cloud Semantic Segmentation [4.02235104503587]
  In this paper, we harness the information from the three-dimensional representation to proficiently capture local features. A GPU-based KDTree allows for rapid building, querying, and enhancing projection with straightforward operations. We show that a reduced version of our model not only demonstrates strong competitiveness against full-scale state-of-the-art models but also operates in real-time.
  arXiv  Detail & Related papers  (2024-10-14T13:49:05Z)
• SemiDDM-Weather: A Semi-supervised Learning Framework for All-in-one Adverse Weather Removal [57.52777076116241]
  Adverse weather removal aims to restore clear vision under adverse weather conditions. This paper presents a pioneering semi-supervised all-in-one adverse weather removal framework built on the teacher-student network.
  arXiv  Detail & Related papers  (2024-09-29T12:12:22Z)
• Enhancing Autonomous Vehicle Perception in Adverse Weather through Image Augmentation during Semantic Segmentation Training [0.0]
  We trained encoder-decoder UNet models to perform semantic segmentation augmentations. Models trained on weather data have significantly lower losses than those trained on augmented data in all conditions except for clear days.
  arXiv  Detail & Related papers  (2024-08-14T00:08:28Z)
• CondTSF: One-line Plugin of Dataset Condensation for Time Series Forecasting [22.473436770730657]
  The objective of dataset condensation is to ensure that the model trained with the synthetic dataset can perform comparably to the model trained with full datasets. In classification, the synthetic data is considered well-distilled if the model trained with the full dataset and the model trained with the synthetic dataset yield identical labels for the same input. In TS-forecasting, the effectiveness of synthetic data distillation is determined by the distance between predictions of the two models.
  arXiv  Detail & Related papers  (2024-06-04T09:18:20Z)
• Scaling Laws for Data Filtering -- Data Curation cannot be Compute Agnostic [99.3682210827572]
  Vision-language models (VLMs) are trained for thousands of GPU hours on carefully curated web datasets. Data curation strategies are typically developed agnostic of the available compute for training. We introduce neural scaling laws that account for the non-homogeneous nature of web data.
  arXiv  Detail & Related papers  (2024-04-10T17:27:54Z)
• Robust Monocular Depth Estimation under Challenging Conditions [81.57697198031975]
  State-of-the-art monocular depth estimation approaches are highly unreliable under challenging illumination and weather conditions. We tackle these safety-critical issues with md4all: a simple and effective solution that works reliably under both adverse and ideal conditions.
  arXiv  Detail & Related papers  (2023-08-18T17:59:01Z)
• ClimaX: A foundation model for weather and climate [51.208269971019504]
  ClimaX is a deep learning model for weather and climate science. It can be pre-trained with a self-supervised learning objective on climate datasets. It can be fine-tuned to address a breadth of climate and weather tasks.
  arXiv  Detail & Related papers  (2023-01-24T23:19:01Z)
• ClimART: A Benchmark Dataset for Emulating Atmospheric Radiative Transfer in Weather and Climate Models [13.514499533538789]
  We build a large dataset, ClimART, with more than emph10 million samples from present, pre-industrial, and future climate conditions, based on the Canadian Earth System Model. ClimART poses several methodological challenges for the ML community, such as multiple out-of-distribution test sets, underlying domain physics, and a trade-off between accuracy and inference speed. We also present several novel baselines that indicate shortcomings of datasets and network architectures used in prior work.
  arXiv  Detail & Related papers  (2021-11-29T16:32:31Z)
• Spatio-Temporal Graph Contrastive Learning [49.132528449909316]
  We propose a Spatio-Temporal Graph Contrastive Learning framework (STGCL) to tackle these issues. We elaborate on four types of data augmentations which disturb data in terms of graph structure, time domain, and frequency domain. Our framework is evaluated across three real-world datasets and four state-of-the-art models.
  arXiv  Detail & Related papers  (2021-08-26T16:05:32Z)
• Lidar Light Scattering Augmentation (LISA): Physics-based Simulation of Adverse Weather Conditions for 3D Object Detection [60.89616629421904]
  Lidar-based object detectors are critical parts of the 3D perception pipeline in autonomous navigation systems such as self-driving cars. They are sensitive to adverse weather conditions such as rain, snow and fog due to reduced signal-to-noise ratio (SNR) and signal-to-background ratio (SBR)
  arXiv  Detail & Related papers  (2021-07-14T21:10:47Z)
• WeatherBench: A benchmark dataset for data-driven weather forecasting [17.76377510880905]
  We present a benchmark dataset for data-driven medium-range weather forecasting. We provide data derived from the ERA5 archive that has been processed to facilitate the use in machine learning models. We provide baseline scores from simple linear regression techniques, deep learning models, as well as purely physical forecasting models.
  arXiv  Detail & Related papers  (2020-02-02T19:20:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.