You Don't Need Prompt Engineering Anymore: The Prompting Inversion

• URL: http://arxiv.org/abs/2510.22251v1
• Date: Sat, 25 Oct 2025 11:04:01 GMT
• Title: You Don't Need Prompt Engineering Anymore: The Prompting Inversion
• Abstract summary: "Sculpting" is a constrained, rule-based prompting method designed to improve upon standard CoT.<n>We evaluate three prompting strategies across three OpenAI model generations.
• Score: 0.3946915822335988
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Prompt engineering, particularly Chain-of-Thought (CoT) prompting, significantly enhances LLM reasoning capabilities. We introduce "Sculpting," a constrained, rule-based prompting method designed to improve upon standard CoT by reducing errors from semantic ambiguity and flawed common sense. We evaluate three prompting strategies (Zero Shot, standard CoT, and Sculpting) across three OpenAI model generations (gpt-4o-mini, gpt-4o, gpt-5) using the GSM8K mathematical reasoning benchmark (1,317 problems). Our findings reveal a "Prompting Inversion": Sculpting provides advantages on gpt-4o (97% vs. 93% for standard CoT), but becomes detrimental on gpt-5 (94.00% vs. 96.36% for CoT on full benchmark). We trace this to a "Guardrail-to-Handcuff" transition where constraints preventing common-sense errors in mid-tier models induce hyper-literalism in advanced models. Our detailed error analysis demonstrates that optimal prompting strategies must co-evolve with model capabilities, suggesting simpler prompts for more capable models.

Related papers

• Soft Guidance Starts to Outperform CoT Prompting as LLMs Improve [9.816961500772061]
  Chain-of-Thought (CoT) prompting remains the standard baseline for evaluating models' reasoning abilities.<n>We evaluate several modern mid-sized language models on a math problem-solving task and find that models specialized for reasoning achieve better performance in a simple zero-shot setting.
  arXiv  Detail & Related papers  (2026-08-04T12:22:31Z)
• One Token Away from Collapse: The Fragility of Instruction-Tuned Helpfulness [12.183451602438753]
  We show that simple lexical constraints (banning a single punctuation character or common word) cause instruction-tuned LLMs to collapse their responses.<n>We show that base models show no systematic collapse under identical constraints, with effects that are small, noisy, and bidirectional.
  arXiv  Detail & Related papers  (2026-04-14T17:40:01Z)
• Reasoning Topology Matters: Network-of-Thought for Complex Reasoning Tasks [5.523132953818281]
  Chain-of-Thought (CoT) produces linear traces, while Tree-of-Thought (ToT) performs branching search.<n>We propose Network-of-Thought (NoT), a framework that models reasoning as a directed graph with typed nodes and edges.
  arXiv  Detail & Related papers  (2026-03-21T09:32:28Z)
• Fragile Thoughts: How Large Language Models Handle Chain-of-Thought Perturbations [2.5782420501870296]
  Chain-of-Thought (CoT) prompting has emerged as a foundational technique for eliciting reasoning from Large Language Models (LLMs)<n>This paper presents a comprehensive empirical evaluation of robustness to 5 CoT perturbation types: textitMathError, UnitConversion, Sycophancy, SkippedSteps, and textitExtraSteps.
  arXiv  Detail & Related papers  (2026-02-11T03:11:30Z)
• MERGETUNE: Continued fine-tuning of vision-language models [77.8627788911249]
  Fine-tuning vision-language models (VLMs) often leads to catastrophic forgetting of pretrained knowledge.<n>We introduce a novel paradigm, continued fine-tuning (CFT), which seeks to recover pretrained knowledge after a zero-shot model has already been adapted.
  arXiv  Detail & Related papers  (2026-01-15T15:15:53Z)
• Teaching Language Models to Reason with Tools [73.21700643314917]
  We present emphHint-Engineering, a new data synthesis strategy that strategically injects diverse hints at optimal points within reasoning paths.<n>CoRT significantly enhances efficiency, reducing token usage by approximately 30% for the 32B model and 50% for the 1.5B model.
  arXiv  Detail & Related papers  (2025-10-23T08:41:44Z)
• Prompting Science Report 2: The Decreasing Value of Chain of Thought in Prompting [0.0]
  Chain-of-Thought (CoT) prompting is a technique that encourages a large language model to "think step by step"<n>The effectiveness of CoT prompting can vary greatly depending on the type of task and model.<n>For models designed with explicit reasoning capabilities, CoT prompting often results in only marginal, if any, gains in answer accuracy.
  arXiv  Detail & Related papers  (2025-06-08T13:41:25Z)
• Concise Reasoning, Big Gains: Pruning Long Reasoning Trace with Difficulty-Aware Prompting [28.537281448659634]
  We propose a difficulty-aware prompting (DAP) method to dynamically shorten reasoning traces without performance loss.<n>In experiments, a student model fine-tuned on just 100K of these difficulty-pruned CoT samples outperforms a model distilled on 800K original Long CoT samples.<n>Our method also generalizes well: across 11 diverse benchmarks, the shorter difficulty-aware CoTs achieve equal or better accuracy than Long chains, using far fewer tokens.
  arXiv  Detail & Related papers  (2025-05-26T09:04:44Z)
• VeriThinker: Learning to Verify Makes Reasoning Model Efficient [52.74493506816969]
  Large Reasoning Models excel at complex tasks using Chain-of-Thought (CoT) reasoning.<n>Their tendency to overthinking leads to unnecessarily lengthy reasoning chains.<n>We introduce VeriThinker, a novel approach for CoT compression.
  arXiv  Detail & Related papers  (2025-05-23T14:17:56Z)
• LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters! [53.84130385074551]
  Large reasoning models (LRMs) tackle complex reasoning problems by following long chain-of-thoughts (Long CoT)<n>We find that a Large Language model (LLM) can effectively learn Long CoT reasoning through data-efficient supervised fine-tuning (SFT) and parameter-efficient low-rank adaptation (LoRA)<n>With just 17k long CoT training samples, the Qwen2.5-32B-Instruct model achieves significant improvements on a wide range of math and coding benchmarks.
  arXiv  Detail & Related papers  (2025-02-11T08:48:48Z)
• ReasonFlux: Hierarchical LLM Reasoning via Scaling Thought Templates [51.633266497799745]
  hierarchical LLM reasoning via scaling thought templates can effectively optimize the reasoning search space.<n>We introduce three innovations: (i) a structured and generic thought template library, containing around 500 high-level thought templates capable of generalizing to similar or relevant reasoning problems; (ii) performing hierarchical reinforcement learning on a sequence of thought templates instead of long CoTs; and (iii) a brand new inference scaling system.
  arXiv  Detail & Related papers  (2025-02-10T18:51:47Z)
• Markovian Transformers for Informative Language Modeling [1.172865818448696]
  Chain-of-Thought (CoT) reasoning often fails to faithfully reflect a language model's underlying decision process.<n>We introduce a Markovian language model framework that can be understood as a reasoning autoencoder.
  arXiv  Detail & Related papers  (2024-04-29T17:36:58Z)
• Large Language Models are Contrastive Reasoners [8.427805316635318]
  We show how contrastive prompting significantly improves the ability of large language models to perform complex reasoning.<n>Our method surpasses zero-shot CoT and few-shot CoT in most arithmetic and commonsense reasoning tasks.
  arXiv  Detail & Related papers  (2024-03-13T03:15:05Z)
• Training Chain-of-Thought via Latent-Variable Inference [30.21067593018967]
  Large language models (LLMs) solve problems more accurately and interpretably when instructed to work out the answer step by step using a chain-of-thought'' prompt. Naively combining CoT with supervised tuning requires supervision not just of the correct answers, but also of detailed rationales that lead to those answers. We propose a fine-tuning strategy that tries to maximize the emphmarginal log-likelihood of generating a correct answer using CoT prompting.
  arXiv  Detail & Related papers  (2023-11-28T17:47:32Z)
• Beyond Chain-of-Thought, Effective Graph-of-Thought Reasoning in Language Models [74.40196814292426]
  We propose Graph-of-Thought (GoT) reasoning, which models human thought processes not only as a chain but also as a graph. GoT captures the non-sequential nature of human thinking and allows for a more realistic modeling of thought processes. We evaluate GoT's performance on a text-only reasoning task and a multimodal reasoning task.
  arXiv  Detail & Related papers  (2023-05-26T02:15:09Z)
• Faithful Chain-of-Thought Reasoning [51.21714389639417]
  Chain-of-Thought (CoT) prompting boosts Language Models' (LM) performance on a gamut of reasoning tasks. We propose Faithful CoT, a reasoning framework involving two stages: Translation and Problem Solving. This guarantees that the reasoning chain provides a faithful explanation of the final answer.
  arXiv  Detail & Related papers  (2023-01-31T03:04:26Z)
• Distilling Reasoning Capabilities into Smaller Language Models [83.66051257039763]
  Step-by-step reasoning approaches like chain of thought (CoT) have proved to be very effective in inducing reasoning capabilities in large language models. However, the success of the CoT approach is fundamentally tied to the model size, and billion parameter-scale models are often needed to get CoT to work. We propose a knowledge distillation approach that leverages the step-by-step CoT reasoning capabilities of larger models and distills these abilities into smaller models.
  arXiv  Detail & Related papers  (2022-12-01T00:39:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.