Brevity is the Soul of Inference Efficiency: Inducing Concision in VLMs via Data Curation

• URL: http://arxiv.org/abs/2606.25432v2
• Date: Mon, 29 Jun 2026 19:08:38 GMT
• Title: Brevity is the Soul of Inference Efficiency: Inducing Concision in VLMs via Data Curation
• Abstract summary: Inference efficiency is typically pursued by shrinking the model.<n>But output length has been inflating, and it is precisely the component the standard toolkit leaves untouched.<n>Here, we argue that brevity is the missing inference-efficiency lever, and that pretraining data curation is a practical way to pull it.
• Score: 6.388637009168626
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Inference efficiency is typically pursued by shrinking the model: distillation, pruning, quantization, and sparse routing each lower per-token cost while treating token count as fixed. But output length has been inflating, and it is precisely the component the standard toolkit leaves untouched. Here, we argue that brevity is the missing inference-efficiency lever, and that pretraining data curation is a practical way to pull it: a model trained on concise, correct data learns to answer in fewer tokens; i.e. it has a lower Cost-of-Pass. We apply our VLM curation pipeline to the MAmmoTH-VL single-image subset, and compare models trained on our curated data, the standard MAmmoTH-VL data, and external open-weight frontier VLMs. On a controlled 20-evaluation set and 14 VLMs at 1B-4B activated parameters, we hold output length fixed with a per-model regression, separating brevity from quality, and price models in FLOPs per correct answer. Curation buys a 35x Cost-of-Pass advantage over the most verbose 4B comparator (Qwen3.5-4B) within $\sim$1 pp of accuracy (0.41 vs 14.58 TFLOPs per correct answer; 0.691 vs 0.704 mean accuracy). Curation also buys a +17.55-percentage-point matched-length accuracy gain over the uncurated baseline that grows with model scale (from +16.7 pp at 1B to +21.2 pp at 4B). This brevity improvement concedes no quality: generic verbosity buys no accuracy at any capability or scale, and the window where reasoning-structured verbosity still earns its tokens shrinks from 4 of 8 capability groups at 2B to 1 of 8 at 4B. Per example, the concise model even reaches correct answers the verbose reasoning model misses, marking reasoning as a distinct curation target rather than something brevity gives up. Inference efficiency in this regime is a tokens-per-correct problem, and brevity is the lever that targets it directly.

Related papers

• Breaking the Token Ceiling: Distilling Smaller, Stronger Byte Models [67.72077328700088]
  We present the first large scale study of overtraining decoder-only dense transformer models varying two dimensions simultaneously.<n>We find that Token-1B models outperform byte models (End-Of-Token-1B and Bytes-1B) in the low-FLOP regime but eventually plateau.<n>By operating over a small vocabulary of 256 bytes instead of on the order of 100K tokens, they circumvent the need for top-kruncation during logit dumping.
  arXiv  Detail & Related papers  (2026-09-11T00:17:22Z)
• Quantization Amplifies Determinism, Not Bias: Scale-Dependent Behavioral Effects of Serving-Time Weight Compression [0.0]
  quantization of mid-sized models is often considered "nearly free"<n>We examine a different question: when several answers are valid, does quantization change what a model chooses to say?<n>We serve three checkpoints (Qwen3-8B/14B/32B) at three weight precisions (W4A16 AWQ, W8A16 FP8-Marlin, and bf16)<n>We collect approximately 71,000 completions paired by prompt and seed across two custom, leak-checked prompt batteries.
  arXiv  Detail & Related papers  (2026-09-07T19:06:31Z)
• Silent Failures in Quantized LLM Reasoning: A Taxonomy-Based Analysis of Hollow Convergence and Failure Mode Shifts [0.0]
  We show that post-training quantization can silently alter how large language models reason even when task accuracy is preserved.<n>We classify 30,000 chain-of-thought outputs from five instruction-tuned LLMs across three quantization precisions.<n>We find that while accuracy is robust across precisions, Hollow Convergence shows a significant size-dependent shift under NF4.
  arXiv  Detail & Related papers  (2026-07-10T21:55:05Z)
• CAVEWOMAN: How Large Language Models Behave Under Linguistic Input and Output Compression [77.41779716950387]
  We present Cavewoman, a two-channel evaluation protocol that scores every generation on task accuracy, realized per-item cost, and reference-text agreement.<n>We evaluate eight models on five datasets at five reduction levels, with both channels measured on the same items.
  arXiv  Detail & Related papers  (2026-06-23T02:56:48Z)
• 20/20 Vision Language Models: A Prescription for Better VLMs through Data Curation Alone [17.84750728317393]
  We show that data curation can lift performance by +11.7pp on average across 20 public VLM benchmarks.<n>At 2B, our curated model surpasses InternVL3.5-2B by 9.9pp at 17x less training compute.<n>Data curation is a high-leverage tool for building better VLMs, reaching near-frontier accuracy at up to 150x less training compute.
  arXiv  Detail & Related papers  (2026-05-12T01:51:03Z)
• Introspection Fine-Tuning (IFT): Training Small LLMs to Introspect [0.0]
  We investigate whether small language models can detect and report on perturbations their own internal activations.<n>We first show that the binary detection paradigm used in prior work is confounded in small models, as steering biases the model toward affirmative responses regardless of the question content.<n>We then introduce emphIntrospection Fine-Tuning (IFT), supervised fine-tuning on sentence-localization examples constructed from the model's own perturbed forward passes.
  arXiv  Detail & Related papers  (2026-05-08T05:30:26Z)
• Self-Calibrating Language Models via Test-Time Discriminative Distillation [18.46710400838861]
  Large language models (LLMs) are systematically overconfident on questions they often answer incorrectly.<n>We introduce $textbfSECL$ ($textbfSE$lf-$textbfC$alibrating $textbfL$anguage Models), a test-time training (TTT) pipeline that exploits this gap as label-free self-supervision.
  arXiv  Detail & Related papers  (2026-03-18T13:28:50Z)
• What Layers When: Learning to Skip Compute in LLMs with Residual Gates [66.23658560048241]
  GateSkip is a residual-stream gating mechanism that enables token-wise layer skipping in decoder-only LMs.<n>Each Attention/MLP branch is equipped with a sigmoid-linear gate that condenses the branch's output before it re-enters the residual stream.
  arXiv  Detail & Related papers  (2025-10-13T16:31:50Z)
• R-Stitch: Dynamic Trajectory Stitching for Efficient Reasoning [80.104336426172]
  Chain-of-thought (CoT) enhances problem-solving ability of large language models.<n>CoT incurs substantial inference cost due to long autoregressive trajectories.<n>We introduce R-Stitch, a training-free hybrid decoding framework.
  arXiv  Detail & Related papers  (2025-07-23T08:14:36Z)
• Sparsity Forcing: Reinforcing Token Sparsity of MLLMs [40.93786579652003]
  We explicitly reinforce token sparsity in well-posed multimodal large language models (MLLMs) through a simple RL-based post-training framework named textitSparsity Forcing.<n>Our method explores the efficiency-accuracy trade-off by running multiple rollouts with different token budgets, where both efficiency (token reduction ratio) and performance (answer correctness) are formulated as joint rewards.
  arXiv  Detail & Related papers  (2025-04-23T01:45:55Z)
• From PEFT to DEFT: Parameter Efficient Finetuning for Reducing Activation Density in Transformers [52.199303258423306]
  We propose a novel density loss that encourages higher activation sparsity in pre-trained models. Our proposed method, textbfDEFT, can consistently reduce activation density by up to textbf44.94% on RoBERTa$_mathrmLarge$ and by textbf53.19% (encoder density) and textbf90.60% (decoder density) on Flan-T5$_mathrmXXL$.
  arXiv  Detail & Related papers  (2024-02-02T21:25:46Z)
• Data-Free Dynamic Compression of CNNs for Tractable Efficiency [46.498278084317704]
  structured pruning approaches have shown promise in lowering floating-point operations without substantial drops in accuracy.<n>We propose HASTE (Hashing for Tractable Efficiency), a data-free, plug-and-play convolution module that instantly reduces a network's test-time inference cost without training or fine-tuning.<n>We demonstrate our approach on the popular vision benchmarks CIFAR-10 and ImageNet, where we achieve a 46.72% reduction in FLOPs with only a 1.25% loss in accuracy.
  arXiv  Detail & Related papers  (2023-09-29T13:09:40Z)
• Gradient-Free Structured Pruning with Unlabeled Data [57.999191898036706]
  We propose a gradient-free structured pruning framework that uses only unlabeled data. Up to 40% of the original FLOP count can be reduced with less than a 4% accuracy loss across all tasks considered.
  arXiv  Detail & Related papers  (2023-03-07T19:12:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.