Reinforced Wasserstein Training for Severity-Aware Semantic Segmentation in Autonomous Driving

• URL: http://arxiv.org/abs/2008.04751v1
• Date: Tue, 11 Aug 2020 15:00:41 GMT
• Title: Reinforced Wasserstein Training for Severity-Aware Semantic Segmentation in Autonomous Driving
• Authors: Xiaofeng Liu, Yimeng Zhang, Xiongchang Liu, Song Bai, Site Li, Jane You
• Abstract summary: We develop a training framework to explore the inter-class correlation by defining its ground metric as misclassification severity. Experiments on both CamVid and Cityscapes datasets evidenced the effectiveness of our Wasserstein loss.
• Score: 45.11602128316305
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Semantic segmentation is important for many real-world systems, e.g., autonomous vehicles, which predict the class of each pixel. Recently, deep networks achieved significant progress w.r.t. the mean Intersection-over Union (mIoU) with the cross-entropy loss. However, the cross-entropy loss can essentially ignore the difference of severity for an autonomous car with different wrong prediction mistakes. For example, predicting the car to the road is much more servery than recognize it as the bus. Targeting for this difficulty, we develop a Wasserstein training framework to explore the inter-class correlation by defining its ground metric as misclassification severity. The ground metric of Wasserstein distance can be pre-defined following the experience on a specific task. From the optimization perspective, we further propose to set the ground metric as an increasing function of the pre-defined ground metric. Furthermore, an adaptively learning scheme of the ground matrix is proposed to utilize the high-fidelity CARLA simulator. Specifically, we follow a reinforcement alternative learning scheme. The experiments on both CamVid and Cityscapes datasets evidenced the effectiveness of our Wasserstein loss. The SegNet, ENet, FCN and Deeplab networks can be adapted following a plug-in manner. We achieve significant improvements on the predefined important classes, and much longer continuous playtime in our simulator.

Related papers

• End-to-End Steering for Autonomous Vehicles via Conditional Imitation Co-Learning [1.5020330976600735]
  This work introduces the conditional imitation co-learning (CIC) approach to address this issue. We propose posing the steering regression problem as classification, we use a classification-regression hybrid loss to bridge the gap between regression and classification. Our model is demonstrated to improve autonomous driving success rate in unseen environment by 62% on average compared to the CIL method.
  arXiv  Detail & Related papers  (2024-11-25T06:37:48Z)
• OpenNet: Incremental Learning for Autonomous Driving Object Detection with Balanced Loss [3.761247766448379]
  The proposed method can obtain better performance than that of the existing methods. The Experimental results upon the CODA dataset show that the proposed method can obtain better performance than that of the existing methods.
  arXiv  Detail & Related papers  (2023-11-25T06:02:50Z)
• Semantic-Fused Multi-Granularity Cross-City Traffic Prediction [17.020546413647708]
  We propose a Semantic-Fused Multi-Granularity Transfer Learning model to achieve knowledge transfer across cities with fused semantics at different granularities. In detail, we design a semantic fusion module to fuse various semantics while conserving static spatial dependencies. We conduct extensive experiments on six real-world datasets to verify the effectiveness of our STL model.
  arXiv  Detail & Related papers  (2023-02-23T04:26:34Z)
• Neurosymbolic hybrid approach to driver collision warning [64.02492460600905]
  There are two main algorithmic approaches to autonomous driving systems. Deep learning alone has achieved state-of-the-art results in many areas. But sometimes it can be very difficult to debug if the deep learning model doesn't work.
  arXiv  Detail & Related papers  (2022-03-28T20:29:50Z)
• Towards Optimal Strategies for Training Self-Driving Perception Models in Simulation [98.51313127382937]
  We focus on the use of labels in the synthetic domain alone. Our approach introduces both a way to learn neural-invariant representations and a theoretically inspired view on how to sample the data from the simulator. We showcase our approach on the bird's-eye-view vehicle segmentation task with multi-sensor data.
  arXiv  Detail & Related papers  (2021-11-15T18:37:43Z)
• Improving Generalization of Transfer Learning Across Domains Using Spatio-Temporal Features in Autonomous Driving [45.655433907239804]
  Vehicle simulation can be used to learn in the virtual world, and the acquired skills can be transferred to handle real-world scenarios. These visual elements are intuitively crucial for human decision making during driving. We propose a CNN+LSTM transfer learning framework to extract thetemporal-temporal features representing vehicle dynamics from scenes.
  arXiv  Detail & Related papers  (2021-03-15T03:26:06Z)
• Importance-Aware Semantic Segmentation in Self-Driving with Discrete Wasserstein Training [44.78636414245145]
  We propose to incorporate the importance-aware inter-class correlation in a Wasserstein training framework. We evaluate our method on CamVid and Cityscapes datasets with different backbones.
  arXiv  Detail & Related papers  (2020-10-21T20:43:47Z)
• Understanding Self-Training for Gradual Domain Adaptation [107.37869221297687]
  We consider gradual domain adaptation, where the goal is to adapt an initial classifier trained on a source domain given only unlabeled data that shifts gradually in distribution towards a target domain. We prove the first non-vacuous upper bound on the error of self-training with gradual shifts, under settings where directly adapting to the target domain can result in unbounded error. The theoretical analysis leads to algorithmic insights, highlighting that regularization and label sharpening are essential even when we have infinite data, and suggesting that self-training works particularly well for shifts with small Wasserstein-infinity distance.
  arXiv  Detail & Related papers  (2020-02-26T08:59:40Z)
• Spatiotemporal Relationship Reasoning for Pedestrian Intent Prediction [57.56466850377598]
  Reasoning over visual data is a desirable capability for robotics and vision-based applications. In this paper, we present a framework on graph to uncover relationships in different objects in the scene for reasoning about pedestrian intent. Pedestrian intent, defined as the future action of crossing or not-crossing the street, is a very crucial piece of information for autonomous vehicles.
  arXiv  Detail & Related papers  (2020-02-20T18:50:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.