ISSAFE: Improving Semantic Segmentation in Accidents by Fusing Event-based Data

• URL: http://arxiv.org/abs/2008.08974v2
• Date: Thu, 9 Dec 2021 16:34:50 GMT
• Title: ISSAFE: Improving Semantic Segmentation in Accidents by Fusing Event-based Data
• Authors: Jiaming Zhang, Kailun Yang, Rainer Stiefelhagen
• Abstract summary: We present a rarely addressed task regarding semantic segmentation in accidental scenarios, along with an accident dataset DADA-seg. We propose a novel event-based multi-modal segmentation architecture ISSAFE. Our approach achieves +8.2% mIoU performance gain on the proposed evaluation set, exceeding more than 10 state-of-the-art segmentation methods.
• Score: 34.36975697486129
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Ensuring the safety of all traffic participants is a prerequisite for bringing intelligent vehicles closer to practical applications. The assistance system should not only achieve high accuracy under normal conditions, but obtain robust perception against extreme situations. However, traffic accidents that involve object collisions, deformations, overturns, etc., yet unseen in most training sets, will largely harm the performance of existing semantic segmentation models. To tackle this issue, we present a rarely addressed task regarding semantic segmentation in accidental scenarios, along with an accident dataset DADA-seg. It contains 313 various accident sequences with 40 frames each, of which the time windows are located before and during a traffic accident. Every 11th frame is manually annotated for benchmarking the segmentation performance. Furthermore, we propose a novel event-based multi-modal segmentation architecture ISSAFE. Our experiments indicate that event-based data can provide complementary information to stabilize semantic segmentation under adverse conditions by preserving fine-grain motion of fast-moving foreground (crash objects) in accidents. Our approach achieves +8.2% mIoU performance gain on the proposed evaluation set, exceeding more than 10 state-of-the-art segmentation methods. The proposed ISSAFE architecture is demonstrated to be consistently effective for models learned on multiple source databases including Cityscapes, KITTI-360, BDD and ApolloScape.

Related papers

• Learning Traffic Crashes as Language: Datasets, Benchmarks, and What-if Causal Analyses [76.59021017301127]
  We propose a large-scale traffic crash language dataset, named CrashEvent, summarizing 19,340 real-world crash reports. We further formulate the crash event feature learning as a novel text reasoning problem and further fine-tune various large language models (LLMs) to predict detailed accident outcomes. Our experiments results show that our LLM-based approach not only predicts the severity of accidents but also classifies different types of accidents and predicts injury outcomes.
  arXiv  Detail & Related papers  (2024-06-16T03:10:16Z)
• DeepAccident: A Motion and Accident Prediction Benchmark for V2X Autonomous Driving [76.29141888408265]
  We propose a large-scale dataset containing diverse accident scenarios that frequently occur in real-world driving. The proposed DeepAccident dataset includes 57K annotated frames and 285K annotated samples, approximately 7 times more than the large-scale nuScenes dataset.
  arXiv  Detail & Related papers  (2023-04-03T17:37:00Z)
• Abnormal Event Detection via Hypergraph Contrastive Learning [54.80429341415227]
  Abnormal event detection plays an important role in many real applications. In this paper, we study the unsupervised abnormal event detection problem in Attributed Heterogeneous Information Network. A novel hypergraph contrastive learning method, named AEHCL, is proposed to fully capture abnormal event patterns.
  arXiv  Detail & Related papers  (2023-04-02T08:23:20Z)
• Augmenting Ego-Vehicle for Traffic Near-Miss and Accident Classification Dataset using Manipulating Conditional Style Translation [0.3441021278275805]
  There is no difference between accident and near-miss at the time before the accident happened. Our contribution is to redefine the accident definition and re-annotate the accident inconsistency on DADA-2000 dataset together with near-miss. The proposed method integrates two different components: conditional style translation (CST) and separable 3-dimensional convolutional neural network (S3D)
  arXiv  Detail & Related papers  (2023-01-06T22:04:47Z)
• Cognitive Accident Prediction in Driving Scenes: A Multimodality Benchmark [77.54411007883962]
  We propose a Cognitive Accident Prediction (CAP) method that explicitly leverages human-inspired cognition of text description on the visual observation and the driver attention to facilitate model training. CAP is formulated by an attentive text-to-vision shift fusion module, an attentive scene context transfer module, and the driver attention guided accident prediction module. We construct a new large-scale benchmark consisting of 11,727 in-the-wild accident videos with over 2.19 million frames.
  arXiv  Detail & Related papers  (2022-12-19T11:43:02Z)
• Towards Robust Semantic Segmentation of Accident Scenes via Multi-Source Mixed Sampling and Meta-Learning [29.74171323437029]
  We propose a Multi-source Meta-learning Unsupervised Domain Adaptation framework, to improve the generalization of segmentation transformers to extreme accident scenes. Our approach achieves a mIoU score of 46.97% on the DADA-seg benchmark, surpassing the previous state-of-the-art model by more than 7.50%.
  arXiv  Detail & Related papers  (2022-03-19T21:18:54Z)
• Exploring Event-driven Dynamic Context for Accident Scene Segmentation [33.20305129155542]
  Most of the critical scenes of traffic accidents are extremely dynamic and previously unseen. We propose to extract dynamic context from event-based data with a higher temporal resolution. Our approach achieves +8.2% performance gain on the proposed accident dataset.
  arXiv  Detail & Related papers  (2021-12-09T16:00:30Z)
• Learning Constraints and Descriptive Segmentation for Subevent Detection [74.48201657623218]
  We propose an approach to learning and enforcing constraints that capture dependencies between subevent detection and EventSeg prediction. We adopt Rectifier Networks for constraint learning and then convert the learned constraints to a regularization term in the loss function of the neural model.
  arXiv  Detail & Related papers  (2021-09-13T20:50:37Z)
• FSNet: A Failure Detection Framework for Semantic Segmentation [9.453250122818808]
  We propose a failure detection framework to identify pixel-level misclassification. We do so by exploiting internal features of the segmentation model and training it simultaneously with a failure detection network. We evaluate the proposed approach against state-of-the-art methods and achieve 12.30%, 9.46%, and 9.65% performance improvement.
  arXiv  Detail & Related papers  (2021-08-19T15:26:52Z)
• Global Feature Aggregation for Accident Anticipation [8.57961305383434]
  We propose a novel Feature Aggregation (FA) block that refines each object's features by computing a weighted sum of the features of all objects in a frame. We use FA block along with Long Short Term Memory (LSTM) network to anticipate accidents in the video sequences.
  arXiv  Detail & Related papers  (2020-06-16T06:17:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.