Identification of Potentially Misclassified Crash Narratives using Machine Learning (ML) and Deep Learning (DL)

• URL: http://arxiv.org/abs/2507.03066v1
• Date: Thu, 03 Jul 2025 17:26:27 GMT
• Title: Identification of Potentially Misclassified Crash Narratives using Machine Learning (ML) and Deep Learning (DL)
• Abstract summary: This research investigates the efficacy of machine learning (ML) and deep learning (DL) methods in detecting misclassified intersection-related crashes in police-reported narratives.<n>Model performance was systematically validated against expert reviews of potentially misclassified narratives.<n>We conclude that hybrid approaches combining automated classification with targeted expert review offer a practical methodology for improving crash data quality.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This research investigates the efficacy of machine learning (ML) and deep learning (DL) methods in detecting misclassified intersection-related crashes in police-reported narratives. Using 2019 crash data from the Iowa Department of Transportation, we implemented and compared a comprehensive set of models, including Support Vector Machine (SVM), XGBoost, BERT Sentence Embeddings, BERT Word Embeddings, and Albert Model. Model performance was systematically validated against expert reviews of potentially misclassified narratives, providing a rigorous assessment of classification accuracy. Results demonstrated that while traditional ML methods exhibited superior overall performance compared to some DL approaches, the Albert Model achieved the highest agreement with expert classifications (73% with Expert 1) and original tabular data (58%). Statistical analysis revealed that the Albert Model maintained performance levels similar to inter-expert consistency rates, significantly outperforming other approaches, particularly on ambiguous narratives. This work addresses a critical gap in transportation safety research through multi-modal integration analysis, which achieved a 54.2% reduction in error rates by combining narrative text with structured crash data. We conclude that hybrid approaches combining automated classification with targeted expert review offer a practical methodology for improving crash data quality, with substantial implications for transportation safety management and policy development.

Related papers

• Detection and Interpretability Analysis of Quotation Errors by Large Language Models [3.806421007129287]
  This paper proposes the task of 'automated detection of quotation errors'<n>The fine-tuning approach for large language models (LLMs) is applied to the task of automated detection of quotation errors.<n> interpretability analysis is conducted on the model's output results.
  arXiv  Detail & Related papers  (2026-06-07T12:01:48Z)
• AI Generated Text Detection [0.0]
  This paper presents an evaluation of AI text detection methods, including both traditional machine learning models and transformer-based architectures.<n>We utilize two datasets, HC3 and DAIGT v2, to build a unified benchmark and apply a topic-based data split to prevent information leakage.<n>Results indicate that contextual modeling is significantly superior to lexical features and highlight the importance of mitigating topic memorization.
  arXiv  Detail & Related papers  (2026-01-07T11:18:10Z)
• AutoMalDesc: Large-Scale Script Analysis for Cyber Threat Research [81.04845910798387]
  Generating natural language explanations for threat detections remains an open problem in cybersecurity research.<n>We present AutoMalDesc, an automated static analysis summarization framework that operates independently at scale.<n>We publish our complete dataset of more than 100K script samples, including annotated seed (0.9K) datasets, along with our methodology and evaluation framework.
  arXiv  Detail & Related papers  (2025-11-17T13:05:25Z)
• Domain-Adapted Pre-trained Language Models for Implicit Information Extraction in Crash Narratives [6.91741018994547]
  We study whether compact open-source language models can support reasoning-intensive extraction from crash narratives.<n>We apply fine-tuning techniques to inject task-specific knowledge to LLMs with Low-Rank Adaption (LoRA) and BERT.<n>Further analysis reveals that the fine-tuned PLMs can capture richer narrative details and even correct some mislabeled annotations in the dataset.
  arXiv  Detail & Related papers  (2025-10-10T14:45:07Z)
• Abex-rat: Synergizing Abstractive Augmentation and Adversarial Training for Classification of Occupational Accident Reports [5.58730646214246]
  ABEX-RAT is a novel framework that synergizes generative data augmentation with robust adversarial training.<n>We show that ABEX-RAT achieves new state-of-the-art performance, reaching a macro-F1 score of 90.32%.
  arXiv  Detail & Related papers  (2025-09-02T08:22:59Z)
• Predicting and Explaining Traffic Crash Severity Through Crash Feature Selection [1.0941365324532635]
  This study introduces a curated dataset of more than 3 million people involved in accidents in Ohio over six years-2022.<n>The primary contribution is a transparent and reproducible methodology that combines Automated Machine Learning (AutoML) and explainable artificial intelligence (AI) to identify and interpret key risk factors associated with severe crashes.<n>Key features spanned demographic, environmental, vehicle, human and operational categories, including location type posted speed, minimum occupant age, and pre-crash action.
  arXiv  Detail & Related papers  (2025-08-15T14:31:26Z)
• DATABench: Evaluating Dataset Auditing in Deep Learning from an Adversarial Perspective [59.66984417026933]
  We introduce a novel taxonomy, classifying existing methods based on their reliance on internal features (IF) (inherent to the data) versus external features (EF) (artificially introduced for auditing)<n>We formulate two primary attack types: evasion attacks, designed to conceal the use of a dataset, and forgery attacks, intending to falsely implicate an unused dataset.<n>Building on the understanding of existing methods and attack objectives, we further propose systematic attack strategies: decoupling, removal, and detection for evasion; adversarial example-based methods for forgery.<n>Our benchmark, DATABench, comprises 17 evasion attacks, 5 forgery attacks, and 9
  arXiv  Detail & Related papers  (2025-07-08T03:07:15Z)
• Benchmarking Adversarial Robustness to Bias Elicitation in Large Language Models: Scalable Automated Assessment with LLM-as-a-Judge [0.0]
  Large Language Models (LLMs) have revolutionized artificial intelligence, driving advancements in machine translation, summarization, and conversational agents.<n>Recent studies indicate that LLMs remain vulnerable to adversarial attacks designed to elicit biased responses.<n>This work proposes a scalable benchmarking framework to evaluate LLM robustness against adversarial bias elicitation.
  arXiv  Detail & Related papers  (2025-04-10T16:00:59Z)
• AdvKT: An Adversarial Multi-Step Training Framework for Knowledge Tracing [64.79967583649407]
  Knowledge Tracing (KT) monitors students' knowledge states and simulates their responses to question sequences.<n>Existing KT models typically follow a single-step training paradigm, which leads to significant error accumulation.<n>We propose a novel Adversarial Multi-Step Training Framework for Knowledge Tracing (AdvKT) which focuses on the multi-step KT task.
  arXiv  Detail & Related papers  (2025-04-07T03:31:57Z)
• Fake News Detection: Comparative Evaluation of BERT-like Models and Large Language Models with Generative AI-Annotated Data [3.7409402247241643]
  Fake news poses a significant threat to public opinion and social stability in modern society.<n>This study presents a comparative evaluation of BERT-like encoder-only models and autoregressive decoder-only large language models (LLMs) for fake news detection.
  arXiv  Detail & Related papers  (2024-12-18T19:15:17Z)
• Learning Traffic Crashes as Language: Datasets, Benchmarks, and What-if Causal Analyses [76.59021017301127]
  We propose a large-scale traffic crash language dataset, named CrashEvent, summarizing 19,340 real-world crash reports. We further formulate the crash event feature learning as a novel text reasoning problem and further fine-tune various large language models (LLMs) to predict detailed accident outcomes. Our experiments results show that our LLM-based approach not only predicts the severity of accidents but also classifies different types of accidents and predicts injury outcomes.
  arXiv  Detail & Related papers  (2024-06-16T03:10:16Z)
• Memory Consistency Guided Divide-and-Conquer Learning for Generalized Category Discovery [56.172872410834664]
  Generalized category discovery (GCD) aims at addressing a more realistic and challenging setting of semi-supervised learning. We propose a Memory Consistency guided Divide-and-conquer Learning framework (MCDL) Our method outperforms state-of-the-art models by a large margin on both seen and unseen classes of the generic image recognition.
  arXiv  Detail & Related papers  (2024-01-24T09:39:45Z)
• Enhancing Intrusion Detection In Internet Of Vehicles Through Federated Learning [0.0]
  Federated learning allows multiple parties to collaborate and learn a shared model without sharing their raw data. Our paper proposes a federated learning framework for intrusion detection in Internet of Vehicles (IOVs) using the CIC-IDS 2017 dataset.
  arXiv  Detail & Related papers  (2023-11-23T04:04:20Z)
• ESimCSE Unsupervised Contrastive Learning Jointly with UDA Semi-Supervised Learning for Large Label System Text Classification Mode [4.708633772366381]
  The ESimCSE model efficiently learns text vector representations using unlabeled data to achieve better classification results. UDA is trained using unlabeled data through semi-supervised learning methods to improve the prediction performance of the models and stability. adversarial training techniques FGM and PGD are used in the model training process to improve the robustness and reliability of the model.
  arXiv  Detail & Related papers  (2023-04-19T03:44:23Z)
• Discover, Explanation, Improvement: An Automatic Slice Detection Framework for Natural Language Processing [72.14557106085284]
  slice detection models (SDM) automatically identify underperforming groups of datapoints. This paper proposes a benchmark named "Discover, Explain, improve (DEIM)" for classification NLP tasks. Our evaluation shows that Edisa can accurately select error-prone datapoints with informative semantic features.
  arXiv  Detail & Related papers  (2022-11-08T19:00:00Z)
• Vehicle Behavior Prediction and Generalization Using Imbalanced Learning Techniques [1.3381749415517017]
  This paper proposes an interaction-aware prediction model consisting of an LSTM autoencoder and SVM classifier. Evaluations show that the method enhances model performance, resulting in improved classification accuracy.
  arXiv  Detail & Related papers  (2021-09-22T11:21:20Z)
• Can Active Learning Preemptively Mitigate Fairness Issues? [66.84854430781097]
  dataset bias is one of the prevailing causes of unfairness in machine learning. We study whether models trained with uncertainty-based ALs are fairer in their decisions with respect to a protected class. We also explore the interaction of algorithmic fairness methods such as gradient reversal (GRAD) and BALD.
  arXiv  Detail & Related papers  (2021-04-14T14:20:22Z)
• Revisiting LSTM Networks for Semi-Supervised Text Classification via Mixed Objective Function [106.69643619725652]
  We develop a training strategy that allows even a simple BiLSTM model, when trained with cross-entropy loss, to achieve competitive results. We report state-of-the-art results for text classification task on several benchmark datasets.
  arXiv  Detail & Related papers  (2020-09-08T21:55:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.