Integrating Crash Report Mining and LLMs for Bug Localization and Repair: An Industrial Report

• URL: http://arxiv.org/abs/2609.04483v1
• Date: Thu, 03 Sep 2026 21:04:40 GMT
• Title: Integrating Crash Report Mining and LLMs for Bug Localization and Repair: An Industrial Report
• Abstract summary: Analyzing crash-report bugs in large-scale industrial software systems requires substantial maintenance effort.<n>We show that the best-performing configuration localizes up to 71% and correctly repairs 52% of crash bugs on the full dataset.
• Score: 7.575395371401797
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Analyzing crash-report bugs in large-scale industrial software systems requires substantial maintenance effort, particularly in production environments where developers must handle large volumes of crash reports and source code artifacts to localize and fix their root causes. While recent studies have shown that Large Language Models (LLMs) can assist with maintenance tasks, little is known about their effectiveness in supporting developers in analyzing crash-report bugs and repairing bugs associated with groups of crash reports in industrial settings. To address this gap, we investigate whether integrating crash report mining techniques---specifically stack trace clustering and suspicious file and method ranking---with LLMs can support crash localization and repair in production environments. We conduct a retrospective evaluation of five LLMs under four prompt configurations. After that, we chose the best model to run on 38 crash bugs collected from two large Java enterprise systems. We further analyze the structural characteristics and explanatory patterns of LLM-generated responses and assess localization and repair effectiveness through manual validation. Our results show that the best-performing configuration localizes up to 71% and correctly repairs 52% of crash bugs on the full dataset. These findings provide empirical evidence that combining crash report mining with LLM-based repair can effectively support debugging activities in industrial maintenance workflows.

Related papers

• How Do LLMs Read Bug Reports? An Empirical Study of Attention in LLMs for Automated Program Repair [7.1093539424910155]
  This paper presents the first empirical study of attention patterns in Large Language Model (LLM)-based program repair.<n>We analyze 319 real-world Python and Java bugs to study how model attention is distributed across bug report sections.<n>We find that successful repairs are characterized by diffused attention across multiple diagnostic components.
  arXiv  Detail & Related papers  (2026-07-28T15:38:12Z)
• Detect--Repair--Verify for LLM-Generated Code: A Multi-Language, Multi-Granularity Empirical Study [10.18490328199727]
  Large language models can generate runnable software artifacts, but their security remains difficult to evaluate end to end.<n>This study examines that problem through a Detect--Repair--Verify (DRV) workflow, in which vulnerabilities are detected, repaired, and then rechecked with security and functional tests.<n>It addresses four gaps in current evidence: the lack of test-grounded benchmarks for LLM-generated artifacts, limited evidence on pipeline-level effectiveness, unclear reliability of detection reports as repair guidance, and uncertain repair trustworthiness under verification.
  arXiv  Detail & Related papers  (2026-03-24T18:18:30Z)
• JFTA-Bench: Evaluate LLM's Ability of Tracking and Analyzing Malfunctions Using Fault Trees [64.57426735981902]
  In maintenance of complex systems, fault trees are used to locate problems and provide targeted solutions.<n>To enable fault trees stored as images to be directly processed by large language models, we propose a novel textual representation of fault trees.<n>We construct a benchmark for multi-turn dialogue systems that emphasizes robust interaction in complex environments.
  arXiv  Detail & Related papers  (2026-03-24T09:13:43Z)
• Outrunning LLM Cutoffs: A Live Kernel Crash Resolution Benchmark for All [57.23434868678603]
  Live-kBench is an evaluation framework for self-evolving benchmarks that scrapes and evaluates agents on freshly discovered kernel bugs.<n> kEnv is an agent-agnostic crash-resolution environment for kernel compilation, execution, and feedback.<n>Using kEnv, we benchmark three state-of-the-art agents, showing that they resolve 74% of crashes on the first attempt.
  arXiv  Detail & Related papers  (2026-02-02T19:06:15Z)
• InspectCoder: Dynamic Analysis-Enabled Self Repair through interactive LLM-Debugger Collaboration [71.18377595277018]
  Large Language Models (LLMs) frequently generate buggy code with complex logic errors that are challenging to diagnose.<n>We present InspectCoder, the first agentic program repair system that empowers LLMs to actively conduct dynamic analysis via interactive debugger control.
  arXiv  Detail & Related papers  (2025-10-21T06:26:29Z)
• RelRepair: Enhancing Automated Program Repair by Retrieving Relevant Code [11.74568238259256]
  RelRepair retrieves relevant project-specific code to enhance automated program repair.<n>We evaluate RelRepair on two widely studied datasets, Defects4J V1.2 and ManySStuBs4J.
  arXiv  Detail & Related papers  (2025-09-20T14:07:28Z)
• Crash Report Enhancement with Large Language Models: An Empirical Study [21.842377922082104]
  We examine whether large language models can enhance crash reports by adding fault locations, root-cause explanations, and repair suggestions.<n>On a dataset of 492 real-world crash reports, LLM-enhanced reports improve Top-1 problem-localization accuracy from 10.6% to 40.2-43.1%.<n>Both our manual evaluations and LLM-as-a-judge assessment show that Agentic-LLM delivers stronger root-cause explanations and more actionable repair guidance.
  arXiv  Detail & Related papers  (2025-09-16T21:02:57Z)
• Exploring Automatic Cryptographic API Misuse Detection in the Era of LLMs [60.32717556756674]
  This paper introduces a systematic evaluation framework to assess Large Language Models in detecting cryptographic misuses. Our in-depth analysis of 11,940 LLM-generated reports highlights that the inherent instabilities in LLMs can lead to over half of the reports being false positives. The optimized approach achieves a remarkable detection rate of nearly 90%, surpassing traditional methods and uncovering previously unknown misuses in established benchmarks.
  arXiv  Detail & Related papers  (2024-07-23T15:31:26Z)
• Learning Traffic Crashes as Language: Datasets, Benchmarks, and What-if Causal Analyses [76.59021017301127]
  We propose a large-scale traffic crash language dataset, named CrashEvent, summarizing 19,340 real-world crash reports. We further formulate the crash event feature learning as a novel text reasoning problem and further fine-tune various large language models (LLMs) to predict detailed accident outcomes. Our experiments results show that our LLM-based approach not only predicts the severity of accidents but also classifies different types of accidents and predicts injury outcomes.
  arXiv  Detail & Related papers  (2024-06-16T03:10:16Z)
• DebugBench: Evaluating Debugging Capability of Large Language Models [80.73121177868357]
  DebugBench is a benchmark for Large Language Models (LLMs) It covers four major bug categories and 18 minor types in C++, Java, and Python. We evaluate two commercial and four open-source models in a zero-shot scenario.
  arXiv  Detail & Related papers  (2024-01-09T15:46:38Z)
• Exploring Large Language Models in Resolving Environment-Related Crash Bugs: Localizing and Repairing [36.4673637256627]
  Large language models (LLMs) have shown promise in software engineering tasks.<n>We conduct the first comprehensive study to assess the capability of LLMs in resolving real-world environment crash bugs.<n>Our findings reveal that localization is the primary challenge for resolving code crashes, while repair poses a greater challenge for environment crashes.
  arXiv  Detail & Related papers  (2023-12-16T13:41:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.