An Active and Contrastive Learning Framework for Fine-Grained Off-Road Semantic Segmentation

• URL: http://arxiv.org/abs/2202.09002v1
• Date: Fri, 18 Feb 2022 03:16:31 GMT
• Title: An Active and Contrastive Learning Framework for Fine-Grained Off-Road Semantic Segmentation
• Authors: Biao Gao, Xijun Zhao, Huijing Zhao
• Abstract summary: Off-road semantic segmentation with fine-grained labels is necessary for autonomous vehicles to understand driving scenes. Fine-grained semantic segmentation in off-road scenes usually has no unified category definition due to ambiguous nature environments. This research proposes an active and contrastive learning-based method that does not rely on pixel-wise labels.
• Score: 7.035838394813961
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Off-road semantic segmentation with fine-grained labels is necessary for autonomous vehicles to understand driving scenes, as the coarse-grained road detection can not satisfy off-road vehicles with various mechanical properties. Fine-grained semantic segmentation in off-road scenes usually has no unified category definition due to ambiguous nature environments, and the cost of pixel-wise labeling is extremely high. Furthermore, semantic properties of off-road scenes can be very changeable due to various precipitations, temperature, defoliation, etc. To address these challenges, this research proposes an active and contrastive learning-based method that does not rely on pixel-wise labels, but only on patch-based weak annotations for model learning. There is no need for predefined semantic categories, the contrastive learning-based feature representation and adaptive clustering will discover the category model from scene data. In order to actively adapt to new scenes, a risk evaluation method is proposed to discover and select hard frames with high-risk predictions for supplemental labeling, so as to update the model efficiently. Experiments conducted on our self-developed off-road dataset and DeepScene dataset demonstrate that fine-grained semantic segmentation can be learned with only dozens of weakly labeled frames, and the model can efficiently adapt across scenes by weak supervision, while achieving almost the same level of performance as typical fully supervised baselines.

Related papers

• Segment-Level Road Obstacle Detection Using Visual Foundation Model Priors and Likelihood Ratios [4.578773000079989]
  Current road obstacle detection methods assign a score to each pixel and apply a threshold to generate final predictions. We propose a novel method that leverages segment-level features from visual foundation models and likelihood ratios to predict road obstacles directly. By focusing on segments rather than individual pixels, our approach enhances detection accuracy, reduces false positives, and offers increased robustness to scene variability.
  arXiv  Detail & Related papers  (2024-12-07T17:40:20Z)
• Physically Feasible Semantic Segmentation [58.17907376475596]
  State-of-the-art semantic segmentation models are typically optimized in a data-driven fashion, minimizing solely per-pixel or per-segment classification objectives on their training data. This purely data-driven paradigm often leads to absurd segmentations, especially when the domain of input images is shifted from the one encountered during training. Our method, Physically Feasible Semantic (PhyFea), first extracts explicit constraints that govern spatial class relations from the semantic segmentation training set at hand in an offline data-driven fashion, and then enforces a morphological yet differentiable loss that penalizes violations of these constraints during
  arXiv  Detail & Related papers  (2024-08-26T22:39:08Z)
• Learning Off-Road Terrain Traversability with Self-Supervisions Only [2.4316550366482357]
  Estimating the traversability of terrain should be reliable and accurate in diverse conditions for autonomous driving in off-road environments. We introduce a method for learning traversability from images that utilizes only self-supervision and no manual labels.
  arXiv  Detail & Related papers  (2023-05-30T09:51:27Z)
• Learning Context-aware Classifier for Semantic Segmentation [88.88198210948426]
  In this paper, contextual hints are exploited via learning a context-aware classifier. Our method is model-agnostic and can be easily applied to generic segmentation models. With only negligible additional parameters and +2% inference time, decent performance gain has been achieved on both small and large models.
  arXiv  Detail & Related papers  (2023-03-21T07:00:35Z)
• LESS: Label-Efficient Semantic Segmentation for LiDAR Point Clouds [62.49198183539889]
  We propose a label-efficient semantic segmentation pipeline for outdoor scenes with LiDAR point clouds. Our method co-designs an efficient labeling process with semi/weakly supervised learning. Our proposed method is even highly competitive compared to the fully supervised counterpart with 100% labels.
  arXiv  Detail & Related papers  (2022-10-14T19:13:36Z)
• SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption [72.35532598131176]
  We propose SCARF, a technique for contrastive learning, where views are formed by corrupting a random subset of features. We show that SCARF complements existing strategies and outperforms alternatives like autoencoders.
  arXiv  Detail & Related papers  (2021-06-29T08:08:33Z)
• Just Label What You Need: Fine-Grained Active Selection for Perception and Prediction through Partially Labeled Scenes [78.23907801786827]
  We introduce generalizations that ensure that our approach is both cost-aware and allows for fine-grained selection of examples through partially labeled scenes. Our experiments on a real-world, large-scale self-driving dataset suggest that fine-grained selection can improve the performance across perception, prediction, and downstream planning tasks.
  arXiv  Detail & Related papers  (2021-04-08T17:57:41Z)
• OFFSEG: A Semantic Segmentation Framework For Off-Road Driving [6.845371503461449]
  We propose a framework for off-road semantic segmentation called as OFFSEG. Off-road semantic segmentation is challenging due to the presence of uneven terrains, unstructured class boundaries, irregular features and strong textures.
  arXiv  Detail & Related papers  (2021-03-23T09:45:41Z)
• Fine-Grained Off-Road Semantic Segmentation and Mapping via Contrastive Learning [7.965964259208489]
  Road detection or traversability analysis has been a key technique for a mobile robot to traverse complex off-road scenes. understanding scenes with fine-grained labels are needed for off-road robots, as scenes are very diverse. This research proposes a contrastive learning based method to achieve meaningful scene understanding for a robot to traverse off-road.
  arXiv  Detail & Related papers  (2021-03-05T13:23:24Z)
• BoMuDANet: Unsupervised Adaptation for Visual Scene Understanding in Unstructured Driving Environments [54.22535063244038]
  We present an unsupervised adaptation approach for visual scene understanding in unstructured traffic environments. Our method is designed for unstructured real-world scenarios with dense and heterogeneous traffic consisting of cars, trucks, two-and three-wheelers, and pedestrians.
  arXiv  Detail & Related papers  (2020-09-22T08:25:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.