Improving Crash Data Quality with Large Language Models: Evidence from Secondary Crash Narratives in Kentucky

• URL: http://arxiv.org/abs/2508.04399v1
• Date: Wed, 06 Aug 2025 12:41:18 GMT
• Title: Improving Crash Data Quality with Large Language Models: Evidence from Secondary Crash Narratives in Kentucky
• Abstract summary: This study evaluates advanced natural language processing (NLP) techniques to enhance crash data quality by mining crash narratives.<n>Drawing from 16,656 manually reviewed narratives from 2015-2022, with 3,803 confirmed secondary crashes, we compare three model classes.<n>Fine-tuned transformers achieved superior performance, with RoBERTa yielding the highest F1-score (0.90) and accuracy 95%.
• Score: 13.50189012080061
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This study evaluates advanced natural language processing (NLP) techniques to enhance crash data quality by mining crash narratives, using secondary crash identification in Kentucky as a case study. Drawing from 16,656 manually reviewed narratives from 2015-2022, with 3,803 confirmed secondary crashes, we compare three model classes: zero-shot open-source large language models (LLMs) (LLaMA3:70B, DeepSeek-R1:70B, Qwen3:32B, Gemma3:27B); fine-tuned transformers (BERT, DistilBERT, RoBERTa, XLNet, Longformer); and traditional logistic regression as baseline. Models were calibrated on 2015-2021 data and tested on 1,771 narratives from 2022. Fine-tuned transformers achieved superior performance, with RoBERTa yielding the highest F1-score (0.90) and accuracy (95%). Zero-shot LLaMA3:70B reached a comparable F1 of 0.86 but required 139 minutes of inference; the logistic baseline lagged well behind (F1:0.66). LLMs excelled in recall for some variants (e.g., GEMMA3:27B at 0.94) but incurred high computational costs (up to 723 minutes for DeepSeek-R1:70B), while fine-tuned models processed the test set in seconds after brief training. Further analysis indicated that mid-sized LLMs (e.g., DeepSeek-R1:32B) can rival larger counterparts in performance while reducing runtime, suggesting opportunities for optimized deployments. Results highlight trade-offs between accuracy, efficiency, and data requirements, with fine-tuned transformer models balancing precision and recall effectively on Kentucky data. Practical deployment considerations emphasize privacy-preserving local deployment, ensemble approaches for improved accuracy, and incremental processing for scalability, providing a replicable scheme for enhancing crash-data quality with advanced NLP.

Related papers

• TriCalRAG: A Three-Strategy, Retrieval-Augmented Benchmark for On-Premise LLM-Based Root Cause Analysis in AIOps [1.8136615181861766]
  Cloud-hosted large language models (LLMs) are increasingly used for root cause analysis (RCA) in AIOps pipelines.<n>They introduce data privacy risk, network latency, and per-query cost that scale poorly with production log volumes.<n>We presentRAG, a benchmark evaluating open-weights served locally via vLLM against a classical LSTM-based log anomaly detector.
  arXiv  Detail & Related papers  (2026-09-13T19:52:46Z)
• Empirical Evaluation of Membership Inference Attacks on NLP Text Classifiers: A Baseline Study on SST-2 [0.0]
  Membership inference attacks (MIAs) try to determine whether a specific record was used to train a model.<n>This paper presents a benchmark of membership inference vulnerability for text classification on the GLUE SST-2 sentiment dataset.
  arXiv  Detail & Related papers  (2026-09-10T00:53:20Z)
• Learning Reasoning World Models for Parallel Code [26.139975890051925]
  A common remedy is to use coding agents that interact with external tools, but tool calls can be costly and sometimes impractical.<n>We propose Parallel-Code World Models (PCWMs) that aim to predict tool outcomes directly from parallel source code.<n>Our results suggest that reasoning models have the potential to serve as practical substitutes for external tool calls in parallel-coding agents.
  arXiv  Detail & Related papers  (2026-04-22T07:29:23Z)
• Pioneer Agent: Continual Improvement of Small Language Models in Production [6.452260317191361]
  Small language models are attractive for production deployment due to their low cost, fast inference, and ease of specialization.<n>We present Pioneer Agent, a closed-loop system that automates the process of adapting small language models to a specific task.
  arXiv  Detail & Related papers  (2026-04-10T18:13:09Z)
• Qwen3-ASR Technical Report [71.87071808763484]
  We introduce Qwen3-ASR family, which includes two powerful all-in-one speech recognition models and a novel non-autoregressive speech forced alignment model.<n>Qwen3-ASR-1.7B and Qwen3-ASR-0.6B are ASR models that support language identification and ASR for 52 languages and dialects.
  arXiv  Detail & Related papers  (2026-01-29T06:58:13Z)
• Hope Speech Detection in Social Media English Corpora: Performance of Traditional and Transformer Models [2.137286893609844]
  This proposal evaluates traditional machine learning models and fine-tuned transformers for a hope speech dataset as train, development and test set.<n> Transformer models delivered better results, the best model achieved weighted precision of 0.82, weighted recall of 0.80, weighted F1 of 0.79, macro F1 of 0.79, and 0.80 accuracy.
  arXiv  Detail & Related papers  (2025-10-27T17:53:40Z)
• A Benchmark Dataset And LLMs Comparison For NFR Classification With Explainable AI [0.21748200848556345]
  Non-Functional Requirements (NFRs) play a critical role in determining the overall quality and user satisfaction of software systems.<n>We collected NFRs from various Project Charters and Open Source Software Documentation.<n>We categorized NFRs into sub-classes and identified needs using widely used Large Language Models.
  arXiv  Detail & Related papers  (2025-10-20T20:45:15Z)
• Enhanced Predictive Modeling for Hazardous Near-Earth Object Detection: A Comparative Analysis of Advanced Resampling Strategies and Machine Learning Algorithms in Planetary Risk Assessment [0.0]
  This study evaluates the performance of several machine learning models for predicting hazardous near-Earth objects (NEOs) through a binary classification framework.<n> RFC and GBC performed the best, both with an impressive F2-score of 0.987 and 0.896, respectively.
  arXiv  Detail & Related papers  (2025-08-20T22:50:00Z)
• OverFill: Two-Stage Models for Efficient Language Model Decoding [68.68408155020568]
  Large language models (LLMs) excel across diverse tasks but face significant deployment challenges due to high inference costs.<n>We propose OverFill, which decouples prefill and decode stages to optimize accuracy-efficiency tradeoffs.<n>Our 3B-to-1B OverFill configuration outperforms 1B pruned models by 83.2%, while the 8B-to-3B configuration improves over 3B pruned models by 79.2% on average.
  arXiv  Detail & Related papers  (2025-08-11T20:07:34Z)
• Goedel-Prover-V2: Scaling Formal Theorem Proving with Scaffolded Data Synthesis and Self-Correction [95.91743732150233]
  Goedel-Prover-V2, a series of open-source language models, set a new state-of-the-art in automated theorem proving.<n>We generate synthetic tasks of increasing difficulty to train the model to master increasingly complex theorems.<n>Goedel-Prover-V2-32B achieves 88.1% on MiniF2F at pass@32 in standard mode and 90.4% in self-correction mode.
  arXiv  Detail & Related papers  (2025-08-05T16:28:22Z)
• Approximating Language Model Training Data from Weights [70.08614275061689]
  We formalize the problem of data approximation from model weights and propose several baselines and metrics.<n>We develop a gradient-based approach that selects the highest-matching data from a large public text corpus.<n>Even when none of the true training data is known, our method is able to locate a small subset of public Web documents.
  arXiv  Detail & Related papers  (2025-06-18T15:26:43Z)
• Improving QA Efficiency with DistilBERT: Fine-Tuning and Inference on mobile Intel CPUs [0.0]
  This study presents an efficient transformer-based question-answering model optimized for deployment on a 13th Gen Intel i7-1355U CPU.<n>The model achieves a validation F1 score of 0.6536 with an average inference time of 0.1208 seconds per question.
  arXiv  Detail & Related papers  (2025-05-28T23:38:11Z)
• Enhancing Grammatical Error Detection using BERT with Cleaned Lang-8 Dataset [0.0]
  This paper presents an improved LLM based model for Grammatical Error Detection (GED) Traditional approach to GED involved hand-designed features, but recently, Neural Networks (NN) have automated the discovery of these features. BERT-base-uncased model gave an impressive performance with an F1 score of 0.91 and accuracy of 98.49% on training data.
  arXiv  Detail & Related papers  (2024-11-23T10:57:41Z)
• AutoFT: Learning an Objective for Robust Fine-Tuning [60.641186718253735]
  Foundation models encode rich representations that can be adapted to downstream tasks by fine-tuning. Current approaches to robust fine-tuning use hand-crafted regularization techniques. We propose AutoFT, a data-driven approach for robust fine-tuning.
  arXiv  Detail & Related papers  (2024-01-18T18:58:49Z)
• Test-Time Adaptation Induces Stronger Accuracy and Agreement-on-the-Line [65.14099135546594]
  Recent test-time adaptation (TTA) methods drastically strengthen the ACL and AGL trends in models, even in shifts where models showed very weak correlations before. Our results show that by combining TTA with AGL-based estimation methods, we can estimate the OOD performance of models with high precision for a broader set of distribution shifts.
  arXiv  Detail & Related papers  (2023-10-07T23:21:25Z)
• Exploring the Value of Pre-trained Language Models for Clinical Named Entity Recognition [6.917786124918387]
  We compare Transformer models that are trained from scratch to fine-tuned BERT-based LLMs. We examine the impact of an additional CRF layer on such models to encourage contextual learning.
  arXiv  Detail & Related papers  (2022-10-23T16:27:31Z)
• Scaling & Shifting Your Features: A New Baseline for Efficient Model Tuning [126.84770886628833]
  Existing finetuning methods either tune all parameters of the pretrained model (full finetuning) or only tune the last linear layer (linear probing) We propose a new parameter-efficient finetuning method termed as SSF, representing that researchers only need to Scale and Shift the deep Features extracted by a pre-trained model to catch up with the performance full finetuning.
  arXiv  Detail & Related papers  (2022-10-17T08:14:49Z)
• AutoDistill: an End-to-End Framework to Explore and Distill Hardware-Efficient Language Models [20.04008357406888]
  We propose AutoDistill, an end-to-end model distillation framework for building hardware-efficient NLP pre-trained models. Experiments on TPUv4i show the finding of seven model architectures with better pre-trained accuracy (up to 3.2% higher) and lower inference latency (up to 1.44x faster) than MobileBERT. By running downstream NLP tasks in the GLUE benchmark, the model distilled for pre-training by AutoDistill with 28.5M parameters achieves an 81.69 average score.
  arXiv  Detail & Related papers  (2022-01-21T04:32:19Z)
• DeBERTa: Decoding-enhanced BERT with Disentangled Attention [119.77305080520718]
  We propose a new model architecture DeBERTa that improves the BERT and RoBERTa models using two novel techniques. We show that these techniques significantly improve the efficiency of model pre-training and the performance of both natural language understanding (NLU) and natural langauge generation (NLG) downstream tasks.
  arXiv  Detail & Related papers  (2020-06-05T19:54:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.