Same Task, Different Work: Prompt-Induced Waste in Coding Agents

• URL: http://arxiv.org/abs/2608.01347v3
• Date: Thu, 06 Aug 2026 05:54:32 GMT
• Title: Same Task, Different Work: Prompt-Induced Waste in Coding Agents
• Abstract summary: Two prompts can request the same code change and produce the same correct patch, yet cause a coding agent to perform radically different kinds and amounts of work.<n>We study this effect in a preregistered benchmark spanning 4,644 valid runs, 24 deterministic coding tasks, seven reasoning models, and two real agent harnesses.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Two prompts can request the same code change and produce the same correct patch, yet cause a coding agent to perform radically different kinds and amounts of work. We study this effect in a preregistered benchmark spanning 4,644 valid runs, 24 deterministic coding tasks, seven reasoning models, and two real agent harnesses. The central finding is that prompt wording does not merely scale total effort; it changes where that effort is spent. Multiple approaches and deep thinking primarily inflate reasoning. Multiple approaches increases reasoning by 2.4x to 7.4x across all six open models and creates about three elaborated but discarded solution branches, while still yielding only one implemented solution and no success gain. Maximum certainty activates a different pathway: repeated verification propagates into extra test runs, tool calls, turns, latency, and context growth. Runs with high redundant verification cost 18x the clean-run median, execute 2.5x more tool calls, and take 3x longer, again without a success gradient. These mechanisms therefore have distinct cost carriers: some prompts are reasoning-heavy and token-borne, while others are tool-heavy and system-borne. Harness design amplifies both effects and changes cost per successful task by 5x to 30x in our setting. The findings survive a frozen holdout, paraphrase tests, a Kimi-K3 replication, and a first-party Claude Sonnet 5 study. In contrast, bounded-efficiency wording preserves diagnosis and final validation while avoiding the measured waste mechanisms. Prompt engineering for coding agents is therefore work design: it determines what the agent thinks through, what it executes, and when it stops.

Related papers

• Agent Mesh: Reliability Primitives for Non-Idempotent Agent Delegation - Identity Adequacy and Evidence Adequacy [0.0]
  We report a failure study of a production agentic software-delivery platform over 147 numbered incidents spanning 81 runs.<n>We quantify the consequences: a loop of fifty-four consecutive successful tool calls no error-rate breaker could see.<n>We derive seven reliability primitives whose enforcement unit is the delegation rather than the message.
  arXiv  Detail & Related papers  (2026-08-26T15:38:21Z)
• Autoresearch with Coding Agents: Generalizers and Metric-Maximizers on Quran Recitation Data [4.191965713559235]
  Coding agents can now be left alone to improve software against a score.<n>In this pattern, the agent receives a dataset, an evaluation script, and one editable file, and iterates without supervision.<n>We ran this loop on a real production task: deciding which Quranic verses appear in a noisy speech-recognition transcript.
  arXiv  Detail & Related papers  (2026-07-20T15:32:09Z)
• Fantastic Adaptive Taxonomies and How to Use Them [100.20614173157708]
  AdaMAST converts execution traces into a compact, evidence-grounded failure taxonomy.<n>No code is hand-authored, no trace human-annotated.<n>In agent-system search, taxonomy-coded diagnoses of failed candidates outperform free-form.
  arXiv  Detail & Related papers  (2026-07-17T17:41:16Z)
• Measuring Harness-Induced Belief Divergence in Multi-Step LLM Agents [0.05371337604556311]
  We show that a software-agent harness can change the agent's multi-step beliefs even when the task, environment, and base LLM are fixed.<n>We introduce a belief-rollout diagnostic that elicits structured K-step trajectories over progress, risk, recoverability, constraints, failure mode, uncertainty, future success, repair cost, and next action under alternative harnesses.
  arXiv  Detail & Related papers  (2026-07-05T22:22:39Z)
• SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks [55.76734816061826]
  We introduce SlopCodeBench, a language-agnostic benchmark comprising 20 problems and 93 checkpoints.<n>We track two trajectory-level quality signals: verbosity, the fraction of redundant or duplicated code, and structural erosion.<n>No agent solves any problem end-to-end across 11 models; the highest checkpoint solve rate is 17.2%.
  arXiv  Detail & Related papers  (2026-03-25T19:26:44Z)
• The Price Reversal Phenomenon: When Cheaper Reasoning Models End Up Costing More [76.93600828673503]
  Listed API pricing is an unreliable proxy for actual cost.<n>Removing thinking token costs reduces ranking reversals by 70%.<n>Our findings call for cost-aware model selection and transparent per-request cost monitoring.
  arXiv  Detail & Related papers  (2026-03-25T06:07:39Z)
• Rerank Before You Reason: Analyzing Reranking Tradeoffs through Effective Token Cost in Deep Search Agents [50.212640395029744]
  We study how to allocate reasoning budget in deep search pipelines.<n>Using the BrowseComp-Plus benchmark, we analyze tradeoffs between model scale, reasoning effort, reranking depth, and total token cost.
  arXiv  Detail & Related papers  (2026-01-20T18:38:35Z)
• CODE: A Contradiction-Based Deliberation Extension Framework for Overthinking Attacks on Retrieval-Augmented Generation [43.85448261466922]
  We propose an end-to-end attack framework named Contradiction-Based Deliberation Extension (CODE)<n> CODE develops a multi-agent architecture to construct poisoning samples that are injected into the knowledge base.<n>Experiments show that CODE causes a 5.32x-24.72x increase in reasoning token consumption, without degrading task performance.
  arXiv  Detail & Related papers  (2026-01-19T14:52:31Z)
• Correct, Concise and Complete: Multi-stage Training For Adaptive Reasoning [11.179446105672461]
  We propose a multi-stage efficient reasoning method that combines supervised fine-tuning and reinforcement learning.<n>Our approach reduces response length by an average of 28% for 8B models and 40% for 32B models.<n>It achieves a superior trade-off compared to more complex state-of-the-art efficient reasoning methods.
  arXiv  Detail & Related papers  (2026-01-06T12:31:51Z)
• Understanding and Steering the Cognitive Behaviors of Reasoning Models at Test-Time [22.9491443902816]
  We study the structure of reasoning trajectories and uncover specialized attention heads that correlate with distinct cognitive behaviors.<n>We propose CREST, a training-free method for Cognitive REasoning Steering at Test-time.<n>CREST adaptively suppresses unproductive reasoning behaviors, yielding both higher accuracy and lower computational cost.
  arXiv  Detail & Related papers  (2025-12-31T02:46:04Z)
• TeaRAG: A Token-Efficient Agentic Retrieval-Augmented Generation Framework [62.66056331998838]
  TeaRAG is a token-efficient agentic RAG framework capable of compressing both retrieval content and reasoning steps.<n>Our reward function evaluates the knowledge sufficiency by a knowledge matching mechanism, while penalizing excessive reasoning steps.
  arXiv  Detail & Related papers  (2025-11-07T16:08:34Z)
• Stop When Enough: Adaptive Early-Stopping for Chain-of-Thought Reasoning [46.106795445750855]
  REFRAIN is a training-free framework that determines when to stop reasoning to mitigate overthinking.<n> REFRAIN reduces token usage by 20-55% while maintaining or improving accuracy compared to standard CoT prompting.
  arXiv  Detail & Related papers  (2025-10-11T08:30:00Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• Abduct, Act, Predict: Scaffolding Causal Inference for Automated Failure Attribution in Multi-Agent Systems [20.846301581161978]
  Failure attribution in multi-agent systems is a critical yet unsolved challenge.<n>Current methods treat this as a pattern recognition task over long conversation logs.<n>A2P Scaffolding transforms failure attribution from pattern recognition into a structured causal inference task.
  arXiv  Detail & Related papers  (2025-09-12T16:51:15Z)
• ConciseHint: Boosting Efficient Reasoning via Continuous Concise Hints during Generation [74.37307916314407]
  We propose a framework dubbed ConciseHint, which continuously encourages the reasoning model to speak concisely.<n>Experiments on the state-of-the-art LRMs, including DeepSeek-R1 and Qwen-3 series, demonstrate that our method can effectively produce concise reasoning.
  arXiv  Detail & Related papers  (2025-06-23T16:20:44Z)
• What makes Reasoning Models Different? Follow the Reasoning Leader for Efficient Decoding [84.42056293290015]
  We analyze the token-level misalignment between reasoning and non-reasoning models.<n>Motivated by the Local Misalignment Diminish, we propose FoReaL-Decoding.<n>On four popular math-reasoning benchmarks, FoReaL-Decoding reduces theoretical FLOPs by 30 to 50% and trims CoT length by up to 40%.
  arXiv  Detail & Related papers  (2025-06-08T05:08:32Z)
• The Price of a Second Thought: On the Evaluation of Reasoning Efficiency in Large Language Models [54.88805865447848]
  We show that instruct models achieve higher efficiency overall, and problem difficulty affects efficiency.<n>We propose COTHINK, a simple two-stage pipeline: an instruct model drafts a brief outline, and a thinking model expands it.<n>On GSM8K, MATH500, and AIME24, COTHINK cuts token usage by 21.1% while keeping accuracy on four thinking models, and remains competitive with strong efficiency baselines.
  arXiv  Detail & Related papers  (2025-05-28T06:24:45Z)
• Runaway is Ashamed, But Helpful: On the Early-Exit Behavior of Large Language Model-based Agents in Embodied Environments [54.67512489842682]
  Large language models (LLMs) have demonstrated strong planning and decision-making capabilities in complex embodied environments.<n>We take a first step toward exploring the early-exit behavior for LLM-based agents.
  arXiv  Detail & Related papers  (2025-05-23T08:23:36Z)
• Adversarial Reasoning for Repair Based on Inferred Program Intent [12.923634025700826]
  We propose an approach called AdverIntent-Agent, based on critique and adversarial reasoning.<n>Our approach is novel to shift the focus from generating multiple APR patches to inferring multiple potential program intents.<n> AdverIntent-Agent was evaluated on two benchmarks: Defects4J 2.0 and HumanEval-Java.
  arXiv  Detail & Related papers  (2025-05-19T11:51:56Z)
• SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning [14.020244011380063]
  SpecReason is a system that accelerates LRM inference.<n>It exploits the semantic flexibility of thinking tokens in preserving final-answer accuracy.<n>It achieves $1.4-3.0times$ speedup over vanilla LRM inference.
  arXiv  Detail & Related papers  (2025-04-10T16:05:19Z)
• When Disagreements Elicit Robustness: Investigating Self-Repair Capabilities under LLM Multi-Agent Disagreements [56.29265568399648]
  We argue that disagreements prevent premature consensus and expand the explored solution space.<n>Disagreements on task-critical steps can derail collaboration depending on the topology of solution paths.
  arXiv  Detail & Related papers  (2025-02-21T02:24:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.