Distilling Reasoning Traces into Advisory Prompts for Software Engineering Tasks

• URL: http://arxiv.org/abs/2608.00437v1
• Date: Sat, 01 Aug 2026 04:32:18 GMT
• Title: Distilling Reasoning Traces into Advisory Prompts for Software Engineering Tasks
• Abstract summary: "Reasoning" or "thinking" modes, exposed as a togglable feature by hybrid reasoning models, do reduce errors.<n>This paper asks if better performance can be achieved without always incurring the cost of reasoning.
• Score: 3.4482431747857674
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Language models are widely used for generating and otherwise processing code (e.g., identifying code hallucinations, possible inputs, or predicting outputs); however, LLMs can make mistakes, which can be serious. One key issue is that models are trained on (still) largely human-written, and thus imperfect, code; it's not easy to find sufficiently large code corpora that are entirely free of bugs. Thus, other inference-time ways of reducing LLM errors, without additional training, are desirable. "Reasoning" or "thinking" modes, exposed as a togglable feature by hybrid reasoning models, do reduce errors; however, reasoning consumes additional resources. This paper asks if better performance can be achieved without always incurring the cost of reasoning. Human students of programming learn to avoid mistakes by (a) identifying them, (b) reflecting upon the cognitive lapses that led to them (essentially, "thinking through" the errors), (c) inferring general rules or lessons from these reflections, and (d) internalizing these lessons into rules. In tutorial sessions with an instructor, this is a common Socratic interaction. Examples of such internalizable rules might include the nugget "Before coding, restate the requirements to clarify them." Inspired by this process, this paper describes an approach where we first identify examples in which "thinking mode" in a (low-resource) LLM avoids errors. These errors, and their avoidance via "thinking" in the same LLM, are then examined by a bigger LLM to generate summary explanations; these are then summarized by a large LLM into brief advisory prompts. This approach works on many modest-sized models; in some cases, the "advisory prompts" thus learned can also be gainfully transferred to other models. We also present investigations into the nature of coding errors that language models make, and a characterization of when this approach can be helpful.

Related papers

• Unreliable in Practice? A Comprehensive Study of Errors in LLM-Generated Code [0.8921166277011344]
  Large Language Models (LLMs) are being widely used for coding, with reports indicating that AI now generates an increasing share of production code.<n>We analyzed a corpus of 86,726 code samples that contained compilation or runtime errors.<n>Results show that, although error types vary strongly across languages and models, even the largest models frequently make simple mistakes.
  arXiv  Detail & Related papers  (2026-08-01T13:31:42Z)
• Think Anywhere in Code Generation [89.9273351013764]
  Think-Anywhere is a novel reasoning mechanism that enables LLMs to invoke thinking on-demand during code generation.<n>We show that Think-Anywhere achieves state-of-the-art performance over both existing reasoning methods and recent post-training approaches.<n>Our analysis further reveals that Think-Anywhere enables the model to adaptively invoke reasoning at high-entropy positions, providing enhanced interpretability.
  arXiv  Detail & Related papers  (2026-03-31T16:24:03Z)
• Large Language Model Hacking: Quantifying the Hidden Risks of Using LLMs for Text Annotation [66.84286617519258]
  Large language models are transforming social science research by enabling the automation of labor-intensive tasks like data annotation and text analysis.<n>Such variation can introduce systematic biases and random errors, which propagate to downstream analyses and cause Type I (false positive), Type II (false negative), Type S (wrong sign), or Type M (exaggerated effect) errors.<n>We find that intentional LLM hacking is strikingly simple. By replicating 37 data annotation tasks from 21 published social science studies, we show that, with just a handful of prompt paraphrases, virtually anything can be presented as statistically significant.
  arXiv  Detail & Related papers  (2025-09-10T17:58:53Z)
• Navigating Pitfalls: Evaluating LLMs in Machine Learning Programming Education [2.9248916859490173]
  This study examines the use of Large Language Models in supporting learning in machine learning education.<n>It focuses on the ability of LLMs to identify common errors of practice in machine learning code, and their ability to provide feedback that can guide learning.
  arXiv  Detail & Related papers  (2025-05-23T08:39:58Z)
• Type-Constrained Code Generation with Language Models [51.03439021895432]
  We introduce a type-constrained decoding approach that leverages type systems to guide code generation.<n>For this purpose, we develop novel prefix automata and a search over inhabitable types, forming a sound approach to enforce well-typedness on LLM-generated code.<n>Our approach reduces compilation errors by more than half and significantly increases functional correctness in code synthesis, translation, and repair tasks.
  arXiv  Detail & Related papers  (2025-04-12T15:03:00Z)
• A Deep Dive Into Large Language Model Code Generation Mistakes: What and Why? [9.246899995643918]
  Large Language Models can still generate defective code that deviates from the specification.<n>Seven categories of non-syntactic mistakes were identified through extensive manual analyses.<n>Our evaluation demonstrated that GPT-4 with the ReAct prompting technique can achieve an F1 score of up to 0.65 when identifying reasons for LLM's mistakes.
  arXiv  Detail & Related papers  (2024-11-03T02:47:03Z)
• Can LLMs Learn from Previous Mistakes? Investigating LLMs' Errors to Boost for Reasoning [34.34977150518316]
  textscCoTErrorSet, a new benchmark with 609,432 questions, each designed with both correct and error references. textbfSelf-rethinking prompting guides LLMs to rethink whether they have made similar previous mistakes. textbfMistake tuning involves finetuning models in both correct and incorrect reasoning domains.
  arXiv  Detail & Related papers  (2024-03-29T08:30:34Z)
• Decoding Logic Errors: A Comparative Study on Bug Detection by Students and Large Language Models [5.162225137921625]
  Large language models (LLMs) have recently demonstrated surprising performance for a range of computing tasks. We investigate the performance of two popular LLMs, GPT-3 and GPT-4, for detecting and providing a novice-friendly explanation of logic errors.
  arXiv  Detail & Related papers  (2023-11-27T17:28:33Z)
• Large Language Models can Learn Rules [106.40747309894236]
  We present Hypotheses-to-Theories (HtT), a framework that learns a rule library for reasoning with large language models (LLMs)<n> Experiments on relational reasoning, numerical reasoning and concept learning problems show that HtT improves existing prompting methods.<n>The learned rules are also transferable to different models and to different forms of the same problem.
  arXiv  Detail & Related papers  (2023-10-10T23:07:01Z)
• SelfCheck: Using LLMs to Zero-Shot Check Their Own Step-by-Step Reasoning [55.76083560152823]
  SelfCheck is a general-purpose zero-shot verification schema for recognizing errors in step-by-step reasoning. We test SelfCheck on three datasets (GSM8K, MathQA, and MATH) and find that it successfully recognizes errors and, in turn, increases final answer accuracies.
  arXiv  Detail & Related papers  (2023-08-01T10:31:36Z)
• Language models are weak learners [71.33837923104808]
  We show that prompt-based large language models can operate effectively as weak learners. We incorporate these models into a boosting approach, which can leverage the knowledge within the model to outperform traditional tree-based boosting. Results illustrate the potential for prompt-based LLMs to function not just as few-shot learners themselves, but as components of larger machine learning pipelines.
  arXiv  Detail & Related papers  (2023-06-25T02:39:19Z)
• Do Large Language Models Pay Similar Attention Like Human Programmers When Generating Code? [10.249771123421432]
  We investigate whether Large Language Models (LLMs) attend to the same parts of a task description as human programmers during code generation. We manually analyzed 211 incorrect code snippets and found five attention patterns that can be used to explain many code generation errors. Our findings highlight the need for human-aligned LLMs for better interpretability and programmer trust.
  arXiv  Detail & Related papers  (2023-06-02T00:57:03Z)
• Rethinking with Retrieval: Faithful Large Language Model Inference [91.66406351103484]
  We propose a novel post-processing approach, rethinking with retrieval (RR) RR retrieves relevant external knowledge based on the reasoning steps obtained from the chain-of-thought prompting. We evaluate the effectiveness of RR through extensive experiments with GPT-3 on three complex reasoning tasks.
  arXiv  Detail & Related papers  (2022-12-31T22:35:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.