Accurate Coverage Metrics for Compiler-Generated Debugging Information

• URL: http://arxiv.org/abs/2402.04811v1
• Date: Wed, 7 Feb 2024 13:01:28 GMT
• Title: Accurate Coverage Metrics for Compiler-Generated Debugging Information
• Authors: J. Ryan Stinnett, Stephen Kell
• Abstract summary: Many tools rely on compiler-produced metadata to present a source-language view of program states. Current approaches for measuring the extent of coverage of local variables are based on crude assumptions. We propose some new metrics, computable by our tools, which could serve as motivation for language implementations to improve debug quality.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Many debugging tools rely on compiler-produced metadata to present a source-language view of program states, such as variable values and source line numbers. While this tends to work for unoptimised programs, current compilers often generate only partial debugging information in optimised programs. Current approaches for measuring the extent of coverage of local variables are based on crude assumptions (for example, assuming variables could cover their whole parent scope) and are not comparable from one compilation to another. In this work, we propose some new metrics, computable by our tools, which could serve as motivation for language implementations to improve debugging quality.

Related papers

• MdEval: Massively Multilingual Code Debugging [37.48700033342978]
  We propose the first massively multilingual debug benchmark, which includes 3.6K test samples of 18 programming languages. We introduce the instruction corpora MDEVAL-INSTRUCT by injecting bugs into the correct multilingual queries and solutions. Our experiments on MDEVAL reveal a notable performance gap between open-source models and closed-source LLMs.
  arXiv  Detail & Related papers  (2024-11-04T17:36:40Z)
• LDB: A Large Language Model Debugger via Verifying Runtime Execution Step-by-step [35.76881887942524]
  Large language models (LLMs) are leading significant progress in code generation. In this study, we introduce Large Language Model Debugger (LDB) LDB segments the programs into basic blocks and tracks the values of intermediate variables after each block throughout the runtime execution.
  arXiv  Detail & Related papers  (2024-02-25T00:56:27Z)
• ControlLLM: Augment Language Models with Tools by Searching on Graphs [97.62758830255002]
  We present ControlLLM, a novel framework that enables large language models (LLMs) to utilize multi-modal tools for solving real-world tasks. Our framework comprises three key components: (1) a textittask decomposer that breaks down a complex task into clear subtasks with well-defined inputs and outputs; (2) a textitThoughts-on-Graph (ToG) paradigm that searches the optimal solution path on a pre-built tool graph; and (3) an textitexecution engine with a rich toolbox that interprets the solution path and runs the
  arXiv  Detail & Related papers  (2023-10-26T21:57:21Z)
• Guess & Sketch: Language Model Guided Transpilation [59.02147255276078]
  Learned transpilation offers an alternative to manual re-writing and engineering efforts. Probabilistic neural language models (LMs) produce plausible outputs for every input, but do so at the cost of guaranteed correctness. Guess & Sketch extracts alignment and confidence information from features of the LM then passes it to a symbolic solver to resolve semantic equivalence.
  arXiv  Detail & Related papers  (2023-09-25T15:42:18Z)
• A Static Evaluation of Code Completion by Large Language Models [65.18008807383816]
  Execution-based benchmarks have been proposed to evaluate functional correctness of model-generated code on simple programming problems. static analysis tools such as linters, which can detect errors without running the program, haven't been well explored for evaluating code generation models. We propose a static evaluation framework to quantify static errors in Python code completions, by leveraging Abstract Syntax Trees.
  arXiv  Detail & Related papers  (2023-06-05T19:23:34Z)
• Better Context Makes Better Code Language Models: A Case Study on Function Call Argument Completion [15.068025336990287]
  We show that existing code completion models do not yield good results on our completion task. We query a program analyzer for information relevant to a given function call, and consider ways to provide the analyzer results to different code completion models during inference and training. Our experiments show that providing access to the function implementation and function usages greatly improves the argument completion performance.
  arXiv  Detail & Related papers  (2023-06-01T06:25:58Z)
• Teaching Large Language Models to Self-Debug [62.424077000154945]
  Large language models (LLMs) have achieved impressive performance on code generation. We propose Self- Debugging, which teaches a large language model to debug its predicted program via few-shot demonstrations.
  arXiv  Detail & Related papers  (2023-04-11T10:43:43Z)
• Beyond the C: Retargetable Decompilation using Neural Machine Translation [5.734661402742406]
  We develop a prototype decompiler that is easily retargetable to new languages. We examine the impact of parameters such as tokenization and training data selection on the quality of decompilation. We will release our training data, trained decompilation models, and code to help encourage future research into language-agnostic decompilation.
  arXiv  Detail & Related papers  (2022-12-17T20:45:59Z)
• Fault-Aware Neural Code Rankers [64.41888054066861]
  We propose fault-aware neural code rankers that can predict the correctness of a sampled program without executing it. Our fault-aware rankers can significantly increase the pass@1 accuracy of various code generation models.
  arXiv  Detail & Related papers  (2022-06-04T22:01:05Z)
• Natural Language to Code Translation with Execution [82.52142893010563]
  Execution result--minimum Bayes risk decoding for program selection. We show that it improves the few-shot performance of pretrained code models on natural-language-to-code tasks.
  arXiv  Detail & Related papers  (2022-04-25T06:06:08Z)
• Profile Guided Optimization without Profiles: A Machine Learning Approach [0.0]
  Profile guided optimization is an effective technique for improving the optimization ability of compilers based on dynamic behavior. We present a novel statistical approach to inferring branch probabilities that improves the performance of programs that are compiled without profile guided optimizations.
  arXiv  Detail & Related papers  (2021-12-24T22:49:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.