Brevity Constraints Reverse Performance Hierarchies in Language Models

• URL: http://arxiv.org/abs/2604.00025v1
• Date: Wed, 11 Mar 2026 06:47:41 GMT
• Title: Brevity Constraints Reverse Performance Hierarchies in Language Models
• Abstract summary: On 7.7% of benchmark problems, larger language models underperform smaller ones by 28.4 percentage points despite 10-100x more parameters.<n>We identify the mechanism as spontaneous scale-dependent verbosity that introduces errors through overelaboration.<n>Our results establish that maximizing large model performance requires scale-aware prompt engineering.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Standard evaluation protocols reveal a counterintuitive phenomenon: on 7.7% of benchmark problems spanning five datasets, larger language models underperform smaller ones by 28.4 percentage points despite 10-100x more parameters. Through systematic evaluation of 31 models (0.5B-405B parameters) across 1,485 problems, we identify the mechanism as spontaneous scale-dependent verbosity that introduces errors through overelaboration. Causal intervention experiments demonstrate this reflects correctable prompt design rather than fundamental capability limitations. Constraining large models to produce brief responses improves accuracy by 26 percentage points and reduces performance gaps by up to two-thirds. Most critically, brevity constraints completely reverse performance hierarchies on mathematical reasoning and scientific knowledge benchmarks, with large models achieving 7.7-15.9 percentage point advantages over small models -- direct inversions of the original gaps. These reversals prove large models possess superior latent capabilities that universal prompting masks. We validate findings through three independent contamination tests and demonstrate inverse scaling operates continuously across the full parameter spectrum, with dataset-specific optimal scales ranging from 0.5B to 3.0B parameters. Our results establish that maximizing large model performance requires scale-aware prompt engineering rather than universal evaluation protocols, with immediate implications for deployment: prompt adaptation simultaneously improves accuracy and reduces computational costs.

Related papers

• VibeThinker-3B: Exploring the Frontier of Verifiable Reasoning in Small Language Models [10.513021831152363]
  This report introduces VibeThinker-3B, a compact dense model with 3B parameters developed to investigate how far verifiable reasoning can be pushed within a strictly small-model regime.<n> Experimental evaluations demonstrate that VibeThinker-3B achieves frontier-level performance on highly demanding verifiable tasks.
  arXiv  Detail & Related papers  (2026-06-15T02:57:19Z)
• Simplicity Paradox: Debunking myths about prompting and datasets for LLM evaluation [2.221515121005437]
  Several studies claim better performance with more sophisticated prompting techniques, but do not provide a comprehensive evaluation.<n>We address this gap through a comprehensive empirical study of 8 prompting techniques across 10 multiple-choice question answering (MCQA) datasets.<n>Our findings reveal a striking paradox that baseline prompting consistently outperforms complex reasoning techniques on various benchmarks.
  arXiv  Detail & Related papers  (2026-05-07T21:11:51Z)
• Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels [0.0]
  Large Language Models are routinely compressed via post-training quantization to reduce inference costs and memory footprint for cloud and edge deployment.<n>Existing studies typically compare only two conditions, rely on aggregate bias metrics, and evaluate a single model family.<n>We conduct a controlled empirical study of three instruction-tuned models at five precision levels (BF16 through 3-bit) on 12,148 BBQ bias benchmark items.
  arXiv  Detail & Related papers  (2026-05-02T05:41:47Z)
• Stop Unnecessary Reflection: Training LRMs for Efficient Reasoning with Adaptive Reflection and Length Coordinated Penalty [42.57318973226598]
  ARLCP is a reinforcement learning framework designed to balance reasoning efficiency and solution accuracy.<n>We evaluate our method on five mathematical reasoning benchmarks using DeepSeek-R1-Distill-Qwen-1.5B and DeepSeek-R1-Distill-Qwen-7B models.
  arXiv  Detail & Related papers  (2026-02-12T16:04:00Z)
• Towards a Science of Scaling Agent Systems [79.64446272302287]
  We formalize a definition for agent evaluation and characterize scaling laws as the interplay between agent quantity, coordination structure, modelic, and task properties.<n>We derive a predictive model using coordination metrics, that cross-validated R2=0, enabling prediction on unseen task domains.<n>We identify three effects: (1) a tool-coordination trade-off: under fixed computational budgets, tool-heavy tasks suffer disproportionately from multi-agent overhead, and (2) a capability saturation: coordination yields diminishing or negative returns once single-agent baselines exceed 45%.
  arXiv  Detail & Related papers  (2025-12-09T06:52:21Z)
• Dynamic Template Selection for Output Token Generation Optimization: MLP-Based and Transformer Approaches [0.0]
  Dynamic template selection achieves significant cost reductions without compromising response quality.<n>This work contributes several key elements: formal problem formulation with theoretical grounding in machine learning, four algorithms with corresponding complexity analyses, and extensive empirical validation across production systems.
  arXiv  Detail & Related papers  (2025-11-17T21:00:22Z)
• Systematic Optimization of Open Source Large Language Models for Mathematical Reasoning [1.8254074486719114]
  This paper presents a practical investigation into fine-tuning model parameters for mathematical reasoning tasks.<n>A holistically optimized framework is introduced for five state-of-the-art models on mathematical reasoning tasks.
  arXiv  Detail & Related papers  (2025-09-08T21:31:43Z)
• Enhancement Report Approval Prediction: A Comparative Study of Large Language Models [10.243182983724585]
  Enhancement reports (ERs) serve as a critical communication channel between users and developers, capturing valuable suggestions for software improvement.<n>To address this challenge, enhancement report approval prediction (ERAP) has emerged as a research focus.<n>Recent advancements in large language models (LLM) present new opportunities for enhancing prediction accuracy.
  arXiv  Detail & Related papers  (2025-06-18T03:08:04Z)
• Reasoning Models Are More Easily Gaslighted Than You Think [85.84943447589511]
  We evaluate three state-of-the-art reasoning models, including OpenAI's o4-mini, Claude-3.7-Sonnet and Gemini-2.5-Flash.<n>Our evaluation reveals significant accuracy drops following gaslighting negation prompts.<n>We introduce GaslightingBench-R, a new diagnostic benchmark designed to evaluate reasoning models' susceptibility to defend their belief.
  arXiv  Detail & Related papers  (2025-06-11T12:52:25Z)
• Accelerated Test-Time Scaling with Model-Free Speculative Sampling [58.69141724095398]
  We introduce STAND (STochastic Adaptive N-gram Drafting), a novel model-free speculative decoding approach.<n>We show that STAND reduces inference latency by 60-65% compared to standard autoregressive decoding.<n>As a model-free approach, STAND can be applied to any existing language model without additional training.
  arXiv  Detail & Related papers  (2025-06-05T07:31:18Z)
• EfficientLLM: Efficiency in Large Language Models [64.3537131208038]
  Large Language Models (LLMs) have driven significant progress, yet their growing counts and context windows incur prohibitive compute, energy, and monetary costs.<n>We introduce EfficientLLM, a novel benchmark and the first comprehensive empirical study evaluating efficiency techniques for LLMs at scale.
  arXiv  Detail & Related papers  (2025-05-20T02:27:08Z)
• Ensuring Reproducibility in Generative AI Systems for General Use Cases: A Framework for Regression Testing and Open Datasets [0.0]
  We introduce GPR-bench, a benchmark that operationalizes regression testing for general purpose use cases.<n>We show that newer models generally improve correctness, but the differences are modest and not statistically significant.<n>In contrast, the concise-writing instruction significantly enhances conciseness, demonstrating the effectiveness of prompt engineering.
  arXiv  Detail & Related papers  (2025-05-02T12:31:43Z)
• R-PRM: Reasoning-Driven Process Reward Modeling [53.06844294668382]
  Process Reward Models (PRMs) have emerged as a promising solution by evaluating each reasoning step.<n>Existing PRMs typically output evaluation scores directly, limiting both learning efficiency and evaluation accuracy.<n>We propose Reasoning-Driven Process Reward Modeling (R-PRM)<n>R-PRM generates seed data from limited annotations, effectively bootstrapping our model's reasoning capabilities.
  arXiv  Detail & Related papers  (2025-03-27T09:23:08Z)
• Benchmarking Reasoning Robustness in Large Language Models [76.79744000300363]
  We find significant performance degradation on novel or incomplete data.<n>These findings highlight the reliance on recall over rigorous logical inference.<n>This paper introduces a novel benchmark, termed as Math-RoB, that exploits hallucinations triggered by missing information to expose reasoning gaps.
  arXiv  Detail & Related papers  (2025-03-06T15:36:06Z)
• Predictor-Corrector Enhanced Transformers with Exponential Moving Average Coefficient Learning [73.73967342609603]
  We introduce a predictor-corrector learning framework to minimize truncation errors. We also propose an exponential moving average-based coefficient learning method to strengthen our higher-order predictor. Our model surpasses a robust 3.8B DeepNet by an average of 2.9 SacreBLEU, using only 1/3 parameters.
  arXiv  Detail & Related papers  (2024-11-05T12:26:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.