Reproducing Failures in Fault Signatures

• URL: http://arxiv.org/abs/2309.11004v1
• Date: Wed, 20 Sep 2023 02:14:38 GMT
• Title: Reproducing Failures in Fault Signatures
• Authors: Ashwin Kallingal Joshy, Benjamin Steenhoek, Xiuyuan Guo and Wei Le
• Abstract summary: We generate fault signatures from the failure locations and the original source code to reproduce the faults in small programs. A fault signature aims to contain only sufficient statements that can reproduce the faults. Our evaluation on real-world bugs from Corebench, BugBench, and Manybugs shows that fault signatures can reproduce the fault for the original programs.
• Score: 5.458524825360132
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Software often fails in the field, however reproducing and debugging field failures is very challenging: the failure-inducing input may be missing, and the program setup can be complicated and hard to reproduce by the developers. In this paper, we propose to generate fault signatures from the failure locations and the original source code to reproduce the faults in small executable programs. We say that a fault signature reproduces the fault in the original program if the two failed in the same location, triggered the same error conditions after executing the same selective sequences of failure-inducing statements. A fault signature aims to contain only sufficient statements that can reproduce the faults. That way, it provides some context to inform how a fault is developed and also avoids unnecessary complexity and setups that may block fault diagnosis. To compute fault signatures from the failures, we applied a path-sensitive static analysis tool to generate a path that leads to the fault, and then applied an existing syntactic patching tool to convert the path into an executable program. Our evaluation on real-world bugs from Corebench, BugBench, and Manybugs shows that fault signatures can reproduce the fault for the original programs. Because fault signatures are less complex, automatic test input generation tools generated failure-inducing inputs that could not be generated by using the entire programs. Some failure-inducing inputs can be directly transferred to the original programs. Our experimental data are publicly available at https://doi.org/10.5281/zenodo.5430155.

Related papers

• Combining Logic with Large Language Models for Automatic Debugging and Repair of ASP Programs [1.0650780147044159]
  FormHe is a tool that combines logic-based techniques and Large Language Models to identify and correct issues in Answer Set Programming submissions. We show that FormHe accurately detects faults in 94% of cases and successfully repairs 58% of incorrect submissions.
  arXiv  Detail & Related papers  (2024-10-28T12:30:48Z)
• VDebugger: Harnessing Execution Feedback for Debugging Visual Programs [103.61860743476933]
  We introduce V Debugger, a critic-refiner framework trained to localize and debug visual programs by tracking execution step by step. V Debugger identifies and corrects program errors leveraging detailed execution feedback, improving interpretability and accuracy. Evaluations on six datasets demonstrate V Debugger's effectiveness, showing performance improvements of up to 3.2% in downstream task accuracy.
  arXiv  Detail & Related papers  (2024-06-19T11:09:16Z)
• MOTIF: A tool for Mutation Testing with Fuzzing [3.4742750855568763]
  Mutation testing is a desired practice for embedded software running in safety-critical cyber-physical systems. MOTIF overcomes limitations by leveraging grey-box fuzzing tools to generate unit test cases in C that detect injected faults in mutants.
  arXiv  Detail & Related papers  (2024-06-04T15:12:01Z)
• NExT: Teaching Large Language Models to Reason about Code Execution [50.93581376646064]
  Large language models (LLMs) of code are typically trained on the surface textual form of programs. We propose NExT, a method to teach LLMs to inspect the execution traces of programs and reason about their run-time behavior.
  arXiv  Detail & Related papers  (2024-04-23T01:46:32Z)
• Error Propagation Analysis for Multithreaded Programs: An Empirical Approach [7.1372356952003795]
  Fault injection is a technique to measure the robustness of a program to errors. EPA typically compares the traces of a fault-free (golden) run with those from a faulty run of the program. EPA approaches are unsound for multithreaded programs with non-deterministic golden runs.
  arXiv  Detail & Related papers  (2023-12-28T02:36:02Z)
• Directed Test Program Generation for JIT Compiler Bug Localization [3.626013617212667]
  Bug localization techniques for Just-in-Time (JIT) compilers are based on analyzing the execution behaviors of the target JIT compiler on a set of test programs generated for this purpose. This paper proposes a novel technique for automatic test program generation for JIT compiler bug localization.
  arXiv  Detail & Related papers  (2023-07-17T22:43:02Z)
• Generation Probabilities Are Not Enough: Uncertainty Highlighting in AI Code Completions [54.55334589363247]
  We study whether conveying information about uncertainty enables programmers to more quickly and accurately produce code. We find that highlighting tokens with the highest predicted likelihood of being edited leads to faster task completion and more targeted edits.
  arXiv  Detail & Related papers  (2023-02-14T18:43:34Z)
• Fault-Aware Neural Code Rankers [64.41888054066861]
  We propose fault-aware neural code rankers that can predict the correctness of a sampled program without executing it. Our fault-aware rankers can significantly increase the pass@1 accuracy of various code generation models.
  arXiv  Detail & Related papers  (2022-06-04T22:01:05Z)
• Fast and Accurate Error Simulation for CNNs against Soft Errors [64.54260986994163]
  We present a framework for the reliability analysis of Conal Neural Networks (CNNs) via an error simulation engine. These error models are defined based on the corruption patterns of the output of the CNN operators induced by faults. We show that our methodology achieves about 99% accuracy of the fault effects w.r.t. SASSIFI, and a speedup ranging from 44x up to 63x w.r.t.FI, that only implements a limited set of error models.
  arXiv  Detail & Related papers  (2022-06-04T19:45:02Z)
• Static Prediction of Runtime Errors by Learning to Execute Programs with External Resource Descriptions [31.46148643917194]
  We introduce a real-world dataset and task for predicting runtime errors. We develop an interpreter-inspired architecture with an inductive bias towards mimicking program executions. We show that the model can also predict the location of the error, despite being trained only on labels indicating the presence/absence and kind of error.
  arXiv  Detail & Related papers  (2022-03-07T23:17:17Z)
• Fault-tolerant parity readout on a shuttling-based trapped-ion quantum computer [64.47265213752996]
  We experimentally demonstrate a fault-tolerant weight-4 parity check measurement scheme. We achieve a flag-conditioned parity measurement single-shot fidelity of 93.2(2)%. The scheme is an essential building block in a broad class of stabilizer quantum error correction protocols.
  arXiv  Detail & Related papers  (2021-07-13T20:08:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.