Dataset Creation for Visual Entailment using Generative AI

• URL: http://arxiv.org/abs/2508.11605v1
• Date: Fri, 15 Aug 2025 17:13:41 GMT
• Title: Dataset Creation for Visual Entailment using Generative AI
• Authors: Rob Reijtenbach, Suzan Verberne, Gijs Wijnholds,
• Abstract summary: We present and validate a new synthetic dataset for training visual entailment models.<n>We base our synthetic dataset on the SNLI dataset for textual entailment.<n>We find that synthetic training data only leads to a slight drop in quality on SNLI-VE, with an F-score 0.686 compared to 0.703 when trained on real data.
• Score: 6.055924352838569
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this paper we present and validate a new synthetic dataset for training visual entailment models. Existing datasets for visual entailment are small and sparse compared to datasets for textual entailment. Manually creating datasets is labor-intensive. We base our synthetic dataset on the SNLI dataset for textual entailment. We take the premise text from SNLI as input prompts in a generative image model, Stable Diffusion, creating an image to replace each textual premise. We evaluate our dataset both intrinsically and extrinsically. For extrinsic evaluation, we evaluate the validity of the generated images by using them as training data for a visual entailment classifier based on CLIP feature vectors. We find that synthetic training data only leads to a slight drop in quality on SNLI-VE, with an F-score 0.686 compared to 0.703 when trained on real data. We also compare the quality of our generated training data to original training data on another dataset: SICK-VTE. Again, there is only a slight drop in F-score: from 0.400 to 0.384. These results indicate that in settings with data sparsity, synthetic data can be a promising solution for training visual entailment models.

Related papers

• Generating Realistic Tabular Data with Large Language Models [49.03536886067729]
  Large language models (LLM) have been used for diverse tasks, but do not capture the correct correlation between the features and the target variable. We propose a LLM-based method with three important improvements to correctly capture the ground-truth feature-class correlation in the real data. Our experiments show that our method significantly outperforms 10 SOTA baselines on 20 datasets in downstream tasks.
  arXiv  Detail & Related papers  (2024-10-29T04:14:32Z)
• Data Extrapolation for Text-to-image Generation on Small Datasets [3.7356387436951146]
  We propose a new data augmentation method for text-to-image generation using linear extrapolation. We construct training samples dozens of times larger than the original dataset. Our model achieves FID scores of 7.91, 9.52 and 5.00 on the CUB, Oxford and COCO datasets.
  arXiv  Detail & Related papers  (2024-10-02T15:08:47Z)
• HYPE: Hyperbolic Entailment Filtering for Underspecified Images and Texts [49.21764163995419]
  We introduce HYPerbolic Entailment filtering (HYPE) to extract meaningful and well-aligned data from noisy image-text pair datasets. HYPE not only demonstrates a significant improvement in filtering efficiency but also sets a new state-of-the-art in the DataComp benchmark. This breakthrough showcases the potential of HYPE to refine the data selection process, thereby contributing to the development of more accurate and efficient self-supervised learning models.
  arXiv  Detail & Related papers  (2024-04-26T16:19:55Z)
• DiLM: Distilling Dataset into Language Model for Text-level Dataset Distillation [20.703102374139537]
  We propose a novel text dataset distillation approach called Distilling dataset into Language Model (DiLM) DiLM trains a language model to generate informative synthetic training samples as text data, instead of directly optimizing synthetic samples. Our code will be available at https://github.com/arumaekawa/DiLM.
  arXiv  Detail & Related papers  (2024-03-30T06:40:54Z)
• Retrieval-Augmented Data Augmentation for Low-Resource Domain Tasks [66.87070857705994]
  In low-resource settings, the amount of seed data samples to use for data augmentation is very small. We propose a novel method that augments training data by incorporating a wealth of examples from other datasets. This approach can ensure that the generated data is not only relevant but also more diverse than what could be achieved using the limited seed data alone.
  arXiv  Detail & Related papers  (2024-02-21T02:45:46Z)
• DataComp: In search of the next generation of multimodal datasets [179.79323076587255]
  DataComp is a testbed for dataset experiments centered around a new candidate pool of 12.8 billion image-text pairs from Common Crawl. Our benchmark consists of multiple compute scales spanning four orders of magnitude. In particular, our best baseline, DataComp-1B, enables training a CLIP ViT-L/14 from scratch to 79.2% zero-shot accuracy on ImageNet.
  arXiv  Detail & Related papers  (2023-04-27T11:37:18Z)
• Analysis of Training Object Detection Models with Synthetic Data [0.0]
  This paper attempts to provide a holistic overview of how to use synthetic data for object detection. We analyse aspects of generating the data as well as techniques used to train the models. Experiments are validated on real data and benchmarked to models trained on real data.
  arXiv  Detail & Related papers  (2022-11-29T10:21:16Z)
• ProGen: Progressive Zero-shot Dataset Generation via In-context Feedback [21.168991554983815]
  We propose a progressive zero-shot dataset generation framework, ProGen, to guide the generation of new training data. We show ProGen achieves on-par or superior performance with only 1% synthetic dataset size.
  arXiv  Detail & Related papers  (2022-10-22T02:07:10Z)
• Dataset Distillation by Matching Training Trajectories [75.9031209877651]
  We propose a new formulation that optimize our distilled data to guide networks to a similar state as those trained on real data. Given a network, we train it for several iterations on our distilled data and optimize the distilled data with respect to the distance between the synthetically trained parameters and the parameters trained on real data. Our method handily outperforms existing methods and also allows us to distill higher-resolution visual data.
  arXiv  Detail & Related papers  (2022-03-22T17:58:59Z)
• Omni-supervised Facial Expression Recognition via Distilled Data [120.11782405714234]
  We propose omni-supervised learning to exploit reliable samples in a large amount of unlabeled data for network training. We experimentally verify that the new dataset can significantly improve the ability of the learned FER model. To tackle this, we propose to apply a dataset distillation strategy to compress the created dataset into several informative class-wise images.
  arXiv  Detail & Related papers  (2020-05-18T09:36:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.