DICS: Exploring Data Intrinsic Consistency for Visual Instruction Selection

• URL: http://arxiv.org/abs/2608.30209v3
• Date: Wed, 02 Sep 2026 02:29:37 GMT
• Title: DICS: Exploring Data Intrinsic Consistency for Visual Instruction Selection
• Abstract summary: Data Intrinsic Consistency (DIC) is a self-scoring metric designed to quantify the sample-level inter-component consistency.<n>DICS is an adaptive data selection method that optimize the trade-off between high intra-sample consistency and global distributional diversity.<n>DIC-6M is a 6M-sample multi-modal instruction corpus that enables the largest-scale visual instruction selection study to date.
• Score: 53.457992197673185
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visual instruction tuning is crucial for advancing the vision-language alignment and instruction-following capabilities of Vision-Language Models (VLMs). However, identifying optimal subsets under a fixed ratio constraint from rapidly expanding datasets remains a significant bottleneck. While existing methods largely depend on distribution diversity or heuristic filtering, they often overlook the internal coherence within individual samples. To bridge this gap, we propose Data Intrinsic Consistency (DIC), a self-scoring metric designed to quantify the sample-level inter-component consistency. DIC consists of two modules: Visual Information Consistency (VIC), evaluating the alignment between visual content and instructions, and Response Information Consistency (RIC), assessing response coherence relative to the instruction. Building upon DIC, we introduce Data Intrinsic Consistency Selection (DICS), an adaptive data selection method that optimizes the trade-off between high intra-sample consistency and global distributional diversity under varying data budgets. Extensive experiments demonstrate that DICS consistently outperforms state-of-the-art methods across diverse dataset scales and model architectures, surpassing full-dataset fine-tuning while using only 25% of the LLaVA-1.5-665K data. We further curate DICS-6M, a 6M-sample multi-modal instruction corpus that enables the largest-scale visual instruction selection study to date; remarkably, DICS reaches 94.52\% of the official InternVL3-8B-Instruct performance using less than 25\% of its reported training data. Code can be seen at https://github.com/cqu-student/DICS

Related papers

• Selectivity Drives Efficiency: Dataset Pruning for Visual Place Recognition [52.52131290631774]
  We propose a place-wise dataset pruning framework tailored for visual place recognition (VPR)<n>Our method treats each place as the basic pruning unit and introduces two novel metrics, i.e., intra-place diversity (IPD) and inter-place similarity (IPS)<n>Experiments demonstrate that our method consistently outperforms state-of-the-art DP baselines under different pruning ratios.
  arXiv  Detail & Related papers  (2026-07-16T12:16:00Z)
• Learning Multi-Indicator Weights for Data Selection: A Joint Task-Model Adaptation Framework with Efficient Proxies [50.39041754816285]
  We propose a framework for learning multi-validation weights that jointly adapts data selection to both the downstream task and the specific model.<n>Our approach achieves performance comparable to or exceeding full-dataset tuning while using only 30% of the training samples on GSM8K.
  arXiv  Detail & Related papers  (2026-05-10T17:30:16Z)
• VisNec: Measuring and Leveraging Visual Necessity for Multimodal Instruction Tuning [33.115992843637564]
  We propose a principled data selection framework that measures the marginal contribution of visual input during instruction tuning.<n>By comparing predictive loss with and without visual context, VisNec identifies whether a training instance is vision-critical, redundant, or misaligned.<n>Across 10 benchmarks, training on only 15% of the LLaVA-665K dataset selected by VisNec achieves 100.2% of full-data performance.
  arXiv  Detail & Related papers  (2026-03-01T17:26:02Z)
• OpenDataArena: A Fair and Open Arena for Benchmarking Post-Training Dataset Value [74.80873109856563]
  OpenDataArena (ODA) is a holistic and open platform designed to benchmark the intrinsic value of post-training data.<n>ODA establishes a comprehensive ecosystem comprising four key pillars: (i) a unified training-evaluation pipeline that ensures fair, open comparisons across diverse models; (ii) a multi-dimensional scoring framework that profiles data quality along tens of distinct axes; and (iii) an interactive data lineage explorer to visualize dataset genealogy and dissect component sources.
  arXiv  Detail & Related papers  (2025-12-16T03:33:24Z)
• CoIDO: Efficient Data Selection for Visual Instruction Tuning via Coupled Importance-Diversity Optimization [14.304308878028358]
  Multimodal large language models rely heavily on instruction tuning to align vision and language capabilities.<n>Existing data selection methods aim to select important and diverse subsets, but they often suffer from two critical drawbacks.<n>We introduce CoIDO, a novel dual-objective framework that jointly optimize data importance and diversity to overcome these challenges.
  arXiv  Detail & Related papers  (2025-10-11T09:41:21Z)
• Data Uniformity Improves Training Efficiency and More, with a Convergence Framework Beyond the NTK Regime [24.425649636862122]
  We demonstrate that selecting more uniformly distributed data can improve training efficiency while enhancing performance.<n>Specifically, we establish that more uniform (less biased) distribution leads to a larger minimum pairwise distance between data points.<n>We theoretically show that the approximation error of neural networks decreases as $h_min$ increases.
  arXiv  Detail & Related papers  (2025-06-30T17:58:30Z)
• Less is More: High-value Data Selection for Visual Instruction Tuning [127.38740043393527]
  We propose a high-value data selection approach TIVE, to eliminate redundancy within the visual instruction data and reduce the training cost. Our approach using only about 15% data can achieve comparable average performance to the full-data fine-tuned model across eight benchmarks.
  arXiv  Detail & Related papers  (2024-03-14T16:47:25Z)
• Distribution-Aware Prompt Tuning for Vision-Language Models [20.02599087680773]
  A key to prompt tuning is the feature space alignment between two modalities via learnable vectors with model parameters fixed. Inspired by this observation, we proposed distribution-aware prompt tuning (DAPT) for vision-language models. Our experiments on 11 benchmark datasets demonstrate that our method significantly improves generalizability.
  arXiv  Detail & Related papers  (2023-09-06T23:49:11Z)
• Diversity-Aware Meta Visual Prompting [111.75306320834629]
  We present Diversity-Aware Meta Visual Prompting(DAM-VP), an efficient prompting method for transferring pre-trained models to downstream tasks with frozen backbone. We cluster the downstream dataset into small subsets in a diversity-strapped way, with each subset has its own prompt separately. All the prompts are optimized with a meta-prompt, which is learned across several datasets.
  arXiv  Detail & Related papers  (2023-03-14T17:59:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.