An Agentic Workflow for Detecting Personally Identifiable Information in Crash Narratives

• URL: http://arxiv.org/abs/2604.15369v1
• Date: Wed, 15 Apr 2026 05:03:20 GMT
• Title: An Agentic Workflow for Detecting Personally Identifiable Information in Crash Narratives
• Abstract summary: PII appears sparsely and inconsistently in crash narratives.<n>Existing rule-based approaches often fail to capture context-dependent PII.<n>This study develops and evaluates a locally deployable, agentic workflow for PII detection in crash narratives.
• Score: 25.2699036413582
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Crash narratives in crash reports provide crucial contextual information for traffic safety analysis. Yet, their broader use is hindered by the presence of personally identifiable information (PII), including names, home addresses, and license plate numbers. Because PII appears sparsely and inconsistently in crash narratives, manual detection is not scalable, and existing rule-based approaches often fail to capture context-dependent PII. This study develops and evaluates a locally deployable, agentic workflow for PII detection in crash narratives by leveraging large language models (LLMs). The workflow contains a Hybrid Extractor and a Verifier. The Hybrid Extractor routes structured PII (e.g., phone numbers and email addresses) to a rule-based model (i.e., Presidio) and context-dependent PII (e.g., names, home addresses, and alphanumeric identifiers) to a domain-adapted, fine-tuned LLM. To address ambiguity in challenging categories, the workflow incorporates ensemble LLM extraction and an agentic verification step that filters false detections through evidence-based reasoning. Evaluated on a real-world crash dataset, the agentic workflow achieves strong performance with a precision of 0.82, a recall of 0.94, an F1 of 0.87, and an accuracy of 0.96, outperforming multiple baseline methods. Moreover, the ablation results suggest that ensemble LLM extraction and Verifier offer improved detection for home addresses and alphanumeric identifiers. The workflow runs locally, supporting privacy-sensitive operational settings where external APIs are restricted. This work offers a practical and robust path for scalable, privacy-preserving crash data processing, enabling broader research and safety interventions while safeguarding individual privacy.

Related papers

• From Sequence to Structure: Relational Uncertainty Propagation for LLM Agents [75.69180947909148]
  We propose a trajectory-level UQ framework for large language model (LLM) agents.<n>RuPA represents an execution history as a directed trajectory graph in which reasoning states, tool interactions, and environment feedback are nodes connected by temporal and semantic dependency edges.<n>We evaluate RUPA on representative agent benchmarks, including $$-2, Terminal-Bench-2, and GAIA, using 6 open-source LLMs spanning multiple model families.
  arXiv  Detail & Related papers  (2026-08-17T01:40:14Z)
• Privacy-Preserving Data Drift Detection and Recovery for Large-Scale LLM Applications via Proxy Representations [5.534194634878958]
  ProxyDrift is a framework that identifies and measures drift between production traffic and offline evaluation sets.<n>Our approach operates entirely on non-PII proxy representations.
  arXiv  Detail & Related papers  (2026-08-08T17:15:18Z)
• Beyond Shapley: An Influence-Based Data Auditing Pipeline for LLM Alignment and Evaluation [2.733105115068754]
  We introduce a scalable, inference-only data valuation pipeline that approximates the Shapley value without iterative model retraining.<n>We demonstrate the pipeline's efficacy in sanitizing two heavily vetted alignment datasets.
  arXiv  Detail & Related papers  (2026-07-24T03:31:44Z)
• DREA: Decoupled Reasoning and Exploration Agents for Repository-Level Vulnerability Detection [16.007199928732614]
  We present DREA, a hypothesis-driven framework for repository-level vulnerability detection.<n>Goal-directed context acquisition is the primary source of detection improvement in this design.<n>To support evaluation, we construct RepoPairBench, a repository-grounded benchmark of validated Python vulnerability-fix pairs from real-world projects.
  arXiv  Detail & Related papers  (2026-07-15T04:49:05Z)
• What to Test Next: Interpretable Coverage Gap Discovery in Driving VLMs [52.50210189669399]
  Driving vision-language models (VLMs) must accurately understand scenes across diverse conditions defined by Operational Design Domains (ODDs)<n>We propose SliceScorer, a deterministic scoring rule for missing-slice recommendation.<n>We show that SliceNav surfaces high-risk coverage gaps more effectively than prior slice-discovery methods.
  arXiv  Detail & Related papers  (2026-06-01T03:18:01Z)
• AgentRaft: Automated Detection of Data Over-Exposure in LLM Agents [40.88941407116349]
  We present AgentRaft, the first automated framework for detecting Data Over-Exposure (DOE) risks in Large Language Model (LLM) agents.<n>We evaluate AgentRaft on a testing environment of 6,675 real-world agent tools.
  arXiv  Detail & Related papers  (2026-03-08T09:40:54Z)
• CAPID: Context-Aware PII Detection for Question-Answering Systems [2.538582648751871]
  We propose CAPID, a practical approach that fine-tunes a locally owned small language model (SLM) that filters sensitive information before it is passed to LLMs for QA.<n>Existing datasets do not capture the context-dependent relevance of PII needed to train such a model effectively.<n>Our experiments show that relevance-aware PII detection with a fine-tuned SLM substantially outperforms existing baselines in span, relevance and type accuracy.
  arXiv  Detail & Related papers  (2026-02-10T18:41:31Z)
• The Bitter Lesson of Diffusion Language Models for Agentic Workflows: A Comprehensive Reality Check [54.08619694620588]
  We present a comprehensive evaluation of dLLMs across two distinct agentic paradigms: Embodied Agents and Tool-Calling Agents.<n>Our results on Agentboard and BFCL reveal a "bitter lesson": current dLLMs fail to serve as reliable agentic backbones.
  arXiv  Detail & Related papers  (2026-01-19T11:45:39Z)
• Domain-Adapted Pre-trained Language Models for Implicit Information Extraction in Crash Narratives [6.91741018994547]
  We study whether compact open-source language models can support reasoning-intensive extraction from crash narratives.<n>We apply fine-tuning techniques to inject task-specific knowledge to LLMs with Low-Rank Adaption (LoRA) and BERT.<n>Further analysis reveals that the fine-tuned PLMs can capture richer narrative details and even correct some mislabeled annotations in the dataset.
  arXiv  Detail & Related papers  (2025-10-10T14:45:07Z)
• From Trace to Line: LLM Agent for Real-World OSS Vulnerability Localization [14.474705451897691]
  We present T2L-Agent, a project-level, end-to-end framework that plans its own analysis and narrows scope from modules to exact vulnerable lines.<n>We benchmark T2L-ARVO, a diverse, expert-verified 50-case benchmark spanning five crash families and real-world projects.<n>On T2L-ARVO, T2L-Agent achieves up to 58.0% detection and 54.8% line-level localization, substantially outperforming baselines.
  arXiv  Detail & Related papers  (2025-09-30T22:27:18Z)
• Agent4FaceForgery: Multi-Agent LLM Framework for Realistic Face Forgery Detection [108.5042835056188]
  This work introduces Agent4FaceForgery to address two fundamental problems.<n>How to capture the diverse intents and iterative processes of human forgery creation.<n>How to model the complex, often adversarial, text-image interactions that accompany forgeries in social media.
  arXiv  Detail & Related papers  (2025-09-16T01:05:01Z)
• VulAgent: Hypothesis-Validation based Multi-Agent Vulnerability Detection [55.957275374847484]
  VulAgent is a multi-agent vulnerability detection framework based on hypothesis validation.<n>It implements a semantics-sensitive, multi-view detection pipeline, each aligned to a specific analysis perspective.<n>On average, VulAgent improves overall accuracy by 6.6%, increases the correct identification rate of vulnerable--fixed code pairs by up to 450%, and reduces the false positive rate by about 36%.
  arXiv  Detail & Related papers  (2025-09-15T02:25:38Z)
• Defending against Indirect Prompt Injection by Instruction Detection [109.30156975159561]
  InstructDetector is a novel detection-based approach that leverages the behavioral states of LLMs to identify potential IPI attacks.<n>InstructDetector achieves a detection accuracy of 99.60% in the in-domain setting and 96.90% in the out-of-domain setting, and reduces the attack success rate to just 0.03% on the BIPIA benchmark.
  arXiv  Detail & Related papers  (2025-05-08T13:04:45Z)
• Bridging the Safety Gap: A Guardrail Pipeline for Trustworthy LLM Inferences [18.36319991890607]
  We present Wildflare GuardRail, a guardrail pipeline designed to enhance the safety and reliability of Large Language Model (LLM) inferences.<n>Wildflare GuardRail integrates several core functional modules, including Safety Detector that identifies unsafe inputs and detects hallucinations in model outputs.<n>The lightweight wrappers can address malicious URLs in model outputs in 1.06s per query with 100% accuracy without costly model calls.
  arXiv  Detail & Related papers  (2025-02-12T05:48:57Z)
• Get my drift? Catching LLM Task Drift with Activation Deltas [55.75645403965326]
  Task drift allows attackers to exfiltrate data or influence the LLM's output for other users.<n>We show that a simple linear classifier can detect drift with near-perfect ROC AUC on an out-of-distribution test set.<n>We observe that this approach generalizes surprisingly well to unseen task domains, such as prompt injections, jailbreaks, and malicious instructions.
  arXiv  Detail & Related papers  (2024-06-02T16:53:21Z)
• Online Safety Property Collection and Refinement for Safe Deep Reinforcement Learning in Mapless Navigation [79.89605349842569]
  We introduce the Collection and Refinement of Online Properties (CROP) framework to design properties at training time. CROP employs a cost signal to identify unsafe interactions and use them to shape safety properties. We evaluate our approach in several robotic mapless navigation tasks and demonstrate that the violation metric computed with CROP allows higher returns and lower violations over previous Safe DRL approaches.
  arXiv  Detail & Related papers  (2023-02-13T21:19:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.