20/20 Vision Language Models: A Prescription for Better VLMs through Data Curation Alone

• URL: http://arxiv.org/abs/2605.11405v2
• Date: Wed, 13 May 2026 01:55:26 GMT
• Title: 20/20 Vision Language Models: A Prescription for Better VLMs through Data Curation Alone
• Abstract summary: We show that data curation can lift performance by +11.7pp on average across 20 public VLM benchmarks.<n>At 2B, our curated model surpasses InternVL3.5-2B by 9.9pp at 17x less training compute.<n>Data curation is a high-leverage tool for building better VLMs, reaching near-frontier accuracy at up to 150x less training compute.
• Score: 17.84750728317393
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Data curation has shifted the quality-compute frontier for language-model and contrastive image-text pretraining, but its role for vision-language models (VLMs) is far less established. We ask how far data curation alone can take VLM performance, holding architecture, training recipe, and compute fixed and varying only the training data. Our pipeline, applied to the MAmmoTH-VL single-image subset, lifts performance by +11.7pp on average across 20 public VLM benchmarks (spanning grounding, VQA, OCR/documents, captioning, spatial/3D, counting, charts, math, brand-ID, and multi-image reasoning) and by +11.3pp on average across all nine capability axes of DatBench, our high-fidelity VLM eval suite. At 2B, our curated model surpasses InternVL3.5-2B by 9.9pp at ~17x less training compute and closes the gap to Qwen3-VL-2B to within 1.8pp at ~87x less compute, from pretraining alone. Beyond accuracy, curation delivers four further properties: (1) Reliability: per-capability std across training seeds drops by ~67% and the lift survives a 4k-to-16k context-length sweep; (2) OOD generalization: the 9-eval OOD average rises by +7.2pp, and multi-image BLINK rises by +3.09pp despite single-image-only training, with Visual Correspondence gaining +11.8pp; (3) Behavioral gains beyond benchmarks: across ~1,100 open-ended queries the curated 2B is more honest and more specific than the matched-compute baseline, and more concise and less refusal-prone than a frontier 2B reference; (4) Pareto-dominance on inference cost: at every scale (1B, 2B, 4B) the curated model raises accuracy while lowering response FLOPs vs. the matched-compute baseline, and the curated 4B matches near-frontier accuracy at 3.3x lower response FLOPs than Qwen3-VL-4B. Data curation is a high-leverage tool for building better VLMs, reaching near-frontier accuracy at up to ~150x less training compute.

Related papers

• MOSS-VL Technical Report [66.58420246189313]
  We present MOSS-VL, an open vision-language model family that treats real-time interaction as a first-class capability.<n>It is co-designed across the stack: the language decoder attends to vision only through gated cross-attention.<n> Offline, MOSS-VL-Instruct is competitive at comparable scale and leads temporal-reasoning video sets.
  arXiv  Detail & Related papers  (2026-08-15T05:12:53Z)
• Answer-Conditioned Chain-of-Thought Distillation for Few-Shot Industrial Vision with Small VLMs [0.0]
  We propose answer-conditioned chain-of-thought (CoT) distillation for rapidly adapting small vision-language models to new industrial tasks.<n>A frontier VLM receives each training image along with its correct label and generates a justified visual explanation.<n>We validate on four industrial classification tasks spanning three image modalities using only 18 to 30 labeled images per task.
  arXiv  Detail & Related papers  (2026-07-12T09:23:05Z)
• Brevity is the Soul of Inference Efficiency: Inducing Concision in VLMs via Data Curation [6.388637009168626]
  Inference efficiency is typically pursued by shrinking the model.<n>But output length has been inflating, and it is precisely the component the standard toolkit leaves untouched.<n>Here, we argue that brevity is the missing inference-efficiency lever, and that pretraining data curation is a practical way to pull it.
  arXiv  Detail & Related papers  (2026-06-24T05:50:28Z)
• VisualClaw: A Real-Time, Personalized Agent for the Physical World [76.89211120690028]
  We present VisualClaw, a self-evolving multimodal agent built around two principles.<n>First, hybrid encoding reduces deployment cost by filtering less informative streaming frames with a cascaded gate.<n>Second, skill evolution lets the agent learn from failures, producing skill-bank updates that help future questions.
  arXiv  Detail & Related papers  (2026-06-15T06:58:22Z)
• U-VLM: Hierarchical Vision Language Modeling for Report Generation [20.09433657986766]
  We propose U-VLM, which enables hierarchical vision-language modeling in both training and architecture.<n>U-VLM achieves state-of-the-art performance on CT-RATE and AbdomenAtlas 3.0 using only a 0.1B decoder trained from scratch.
  arXiv  Detail & Related papers  (2026-02-28T05:43:11Z)
• PanoEnv: Exploring 3D Spatial Intelligence in Panoramic Environments with Reinforcement Learning [5.308328605042682]
  360 panoramic images are increasingly used in virtual reality, autonomous driving, and robotics for holistic scene understanding.<n>Current Vision-Language Models (VLMs) struggle with 3D spatial reasoning on Equirectangular Projection (ERP) images due to geometric distortion and limited 3D supervision.<n>We introduce PanoEnv, a large-scale VQA benchmark built from synthetic 3D environments.<n>Our 7B model achieves new state-of-the-art performance, improving overall accuracy to 52.93% (+3.59%) and open-ended accuracy to 14.83% while maintaining structured-task performance.
  arXiv  Detail & Related papers  (2026-02-25T15:12:17Z)
• IVRA: Improving Visual-Token Relations for Robot Action Policy with Training-Free Hint-Based Guidance [35.90972175891154]
  Many Vision-Language-Action (VLA) models flatten image patches into a 1D token sequence, weakening the 2D spatial cues needed for precise manipulation.<n>We introduce a lightweight, training-free method that improves spatial understanding by exploiting affinity hints already available in the model's built-in vision.
  arXiv  Detail & Related papers  (2026-01-22T18:57:13Z)
• STEP3-VL-10B Technical Report [115.89015065130127]
  STEP3-VL-10B is a lightweight foundation model designed to redefine the trade-off between compact efficiency and frontier-level multimodal intelligence.<n>We implement Parallel Coordinated Reasoning (PaCoRe) to scale test-time compute, allocating resources to scalable perceptual reasoning.<n>It records 92.2% on MMBench and 80.11% on MMMU, while excelling in complex reasoning with 94.43% on AIME2025 and 75.95% on MathVision.
  arXiv  Detail & Related papers  (2026-01-14T17:58:24Z)
• Self-Improving VLM Judges Without Human Annotations [74.29324865147838]
  We present a framework to self-train a VLM judge model without any human preference annotations, using only self-synthesized data.<n>Our method improves a Llama-3.2-11B multimodal judge from 0.38 to 0.51 in overall accuracy on Multimodal RewardBench.<n>The overall strength of these human-annotation-free results suggest the potential for a future self-judge that evolves alongside rapidly improving VLM capabilities.
  arXiv  Detail & Related papers  (2025-12-02T20:52:19Z)
• STELAR-VISION: Self-Topology-Aware Efficient Learning for Aligned Reasoning in Vision [24.162895928364062]
  We introduce STELAR-Vision, a training framework for topology-aware reasoning.<n>At its core is TopoAug, a synthetic data pipeline that enriches training with diverse topological structures.<n>On MATH-V and VLM-S2H, STELAR-Vision improves accuracy by 9.7% over its base model and surpasses the larger Qwen2VL-72B-Instruct by 7.3%.
  arXiv  Detail & Related papers  (2025-08-12T07:27:50Z)
• Probing Visual Language Priors in VLMs [51.016683265437536]
  We introduce ViLP, a benchmark featuring deliberately out-of-distribution images.<n>Each question in ViLP is coupled with three potential answers and three corresponding images.<n>We propose a self-improving framework in which models generate new VQA data, then apply pixel-level and semantic corruptions to form "good-bad" image pairs for self-training.
  arXiv  Detail & Related papers  (2024-12-31T17:54:29Z)
• How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites [114.22835695929682]
  InternVL 1.5 is an open-source multimodal large language model (MLLM) It bridges the capability gap between open-source and proprietary commercial models in multimodal understanding.
  arXiv  Detail & Related papers  (2024-04-25T17:59:19Z)
• EfficientVLM: Fast and Accurate Vision-Language Models via Knowledge Distillation and Modal-adaptive Pruning [19.354515754130592]
  We introduce a distilling then pruning framework to compress large vision-language models into smaller, faster, and more accurate ones. We apply our framework to train EfficientVLM, a fast and accurate vision-language model consisting of 6 vision layers, 3 text layers, and 3 cross-modal fusion layers. EfficientVLM retains 98.4% performance of the teacher model and accelerates its inference speed by 2.2x.
  arXiv  Detail & Related papers  (2022-10-14T13:26:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.