Data Selection Through Iterative Self-Filtering for Vision-Language Settings

• URL: http://arxiv.org/abs/2606.23611v1
• Date: Mon, 22 Jun 2026 17:11:15 GMT
• Title: Data Selection Through Iterative Self-Filtering for Vision-Language Settings
• Abstract summary: We propose a novel, bootstrapped method in which a CLIP model is trained on an evolving, self-selected dataset.<n>Our proposed Self-Filtering method iterates between training the model and selecting a subsequently improved data mixture.
• Score: 2.9868893900614406
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The availability of large amounts of clean data is paramount to training neural networks. However, at large scales, manual oversight is impractical, resulting in sizeable datasets that can be very noisy. Attempts to mitigate this obstacle to producing performant vision-language models have so far involved heuristics, curated reference datasets, and using pre-trained models. Here we propose a novel, bootstrapped method in which a CLIP model is trained on an evolving, self-selected dataset. This evolving dataset constitutes a balance of filtered, highly probable clean samples as well as diverse samples from the entire distribution. Our proposed Self-Filtering method iterates between training the model and selecting a subsequently improved data mixture. Training on vision-language datasets filtered by the proposed approach improves downstream performance without the need for additional data or pre-trained models.

Related papers

• SAS: Semantic-aware Sampling for Generative Dataset Distillation [55.27114962330541]
  We introduce a semantic-aware perspective for dataset distillation by leveraging Contrastive Language-Image Pretraining (CLIP) as a semantic prior for post-sampling.<n>Our goal is to obtain distilled datasets that are not only compact but also semantically class-discriminative and diverse.
  arXiv  Detail & Related papers  (2026-05-18T08:05:46Z)
• EvoSelect: Data-Efficient LLM Evolution for Targeted Task Adaptation [79.71802168256542]
  Adapting large language models (LLMs) to a targeted task efficiently and effectively remains a fundamental challenge.<n>One straightforward approach is through an iterative generation-training loop, where candidate data are synthesized through an external generator.<n>We introduce a refined paradigm, namely an iterative generation-selection-training loop, which incorporates a selection step prior to model updates.
  arXiv  Detail & Related papers  (2026-04-28T23:26:16Z)
• DOSE: Data Selection for Multi-Modal LLMs via Off-the-Shelf Models [43.094426671099]
  We study whether off-the-shelf pretrained models that have never seen the target data can be used to select training samples for larger and stronger multimodal models without any task-specific training.<n>We build a joint quality-alignment distribution and apply adaptive weighted sampling to select informative samples while maintaining long-tail diversity.<n>This approach enhances data diversity, enabling models trained on DOSE-filtered data to match or surpass those trained on the full dataset on standard VQA and math benchmarks.
  arXiv  Detail & Related papers  (2026-04-18T12:41:06Z)
• Learnability-Guided Diffusion for Dataset Distillation [51.56484100374058]
  We propose learnability-driven dataset distillation, which constructs synthetic datasets incrementally through successive stages.<n>We introduce Learnability-Guided Diffusion (LGD), which balances training utility for the current model with validity under a reference model to generate curriculum-aligned samples.<n>Our approach reduces redundancy by 39.1%, promotes specialization across training stages, and achieves state-of-the-art results on ImageNet-1K (60.1%), ImageNette (87.2%), and ImageWoof (72.9%)
  arXiv  Detail & Related papers  (2026-04-01T06:04:59Z)
• Dataset Distillation for Pre-Trained Self-Supervised Vision Models [43.50190223507616]
  dataset distillation aims to find a small set of synthetic images such that training a model on them reproduces the performance of the same model trained on a much larger dataset of real samples.<n>We introduce a method of dataset distillation for this task called Linear Gradient Matching.<n>Our method yields synthetic data that outperform all real-image baselines and, remarkably, generalize across pre-trained vision models.
  arXiv  Detail & Related papers  (2025-11-20T18:59:57Z)
• Evaluating Sample Utility for Efficient Data Selection by Mimicking Model Weights [11.237906163959908]
  Multimodal models are trained on large-scale web-crawled datasets.<n>These datasets often contain noise, bias, and irrelevant information.<n>We propose an efficient, model-based approach using the Mimic Score.
  arXiv  Detail & Related papers  (2025-01-12T04:28:14Z)
• Diffusion-Based Neural Network Weights Generation [80.89706112736353]
  D2NWG is a diffusion-based neural network weights generation technique that efficiently produces high-performing weights for transfer learning. Our method extends generative hyper-representation learning to recast the latent diffusion paradigm for neural network weights generation. Our approach is scalable to large architectures such as large language models (LLMs), overcoming the limitations of current parameter generation techniques.
  arXiv  Detail & Related papers  (2024-02-28T08:34:23Z)
• Your Vision-Language Model Itself Is a Strong Filter: Towards High-Quality Instruction Tuning with Data Selection [59.11430077029321]
  We introduce a novel dataset selection method, Self-Filter, for vision-language models (VLMs) In the first stage, we devise a scoring network to evaluate the difficulty of training instructions, which is co-trained with the VLM. In the second stage, we use the trained score net to measure the difficulty of each instruction, select the most challenging samples, and penalize similar samples to encourage diversity.
  arXiv  Detail & Related papers  (2024-02-19T20:08:48Z)
• Self-Evolved Diverse Data Sampling for Efficient Instruction Tuning [47.02160072880698]
  We introduce a self-evolving mechanism that allows the model itself to actively sample subsets that are equally or even more effective. The key to our data sampling technique lies in the enhancement of diversity in the chosen subsets. Extensive experiments across three datasets and benchmarks demonstrate the effectiveness of DiverseEvol.
  arXiv  Detail & Related papers  (2023-11-14T14:10:40Z)
• Data Filtering Networks [67.827994353269]
  We study the problem of learning a data filtering network (DFN) for this second step of filtering a large uncurated dataset. Our key finding is that the quality of a network for filtering is distinct from its performance on downstream tasks. Based on our insights, we construct new data filtering networks that induce state-of-the-art image-text datasets.
  arXiv  Detail & Related papers  (2023-09-29T17:37:29Z)
• Self-Distillation for Further Pre-training of Transformers [83.84227016847096]
  We propose self-distillation as a regularization for a further pre-training stage. We empirically validate the efficacy of self-distillation on a variety of benchmark datasets for image and text classification tasks.
  arXiv  Detail & Related papers  (2022-09-30T02:25:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.