First Train to Generate, then Generate to Train: UnitedSynT5 for Few-Shot NLI

• URL: http://arxiv.org/abs/2412.09263v2
• Date: Fri, 13 Dec 2024 06:28:11 GMT
• Title: First Train to Generate, then Generate to Train: UnitedSynT5 for Few-Shot NLI
• Authors: Sourav Banerjee, Anush Mahajan, Ayushi Agarwal, Eishkaran Singh,
• Abstract summary: We present UnitedSynT5, an advanced extension of Entailment Few-Shot Learning (EFL) We use a T5-based generator to synthesize additional premise-hypothesis pairs, which are rigorously cleaned and integrated into the training data. We train a GTR-T5-XL model on this expanded dataset, achieving a new benchmark of 94.7% accuracy on the SNLI dataset, 94.0% accuracy on the E-SNLI dataset, and 92.6% accuracy on the MultiNLI dataset, surpassing the previous SOTA models.
• Score: 1.2642388972233847
• License:
• Abstract: Natural Language Inference (NLI) tasks require identifying the relationship between sentence pairs, typically classified as entailment, contradiction, or neutrality. While the current state-of-the-art (SOTA) model, Entailment Few-Shot Learning (EFL), achieves a 93.1% accuracy on the Stanford Natural Language Inference (SNLI) dataset, further advancements are constrained by the dataset's limitations. To address this, we propose a novel approach leveraging synthetic data augmentation to enhance dataset diversity and complexity. We present UnitedSynT5, an advanced extension of EFL that leverages a T5-based generator to synthesize additional premise-hypothesis pairs, which are rigorously cleaned and integrated into the training data. These augmented examples are processed within the EFL framework, embedding labels directly into hypotheses for consistency. We train a GTR-T5-XL model on this expanded dataset, achieving a new benchmark of 94.7% accuracy on the SNLI dataset, 94.0% accuracy on the E-SNLI dataset, and 92.6% accuracy on the MultiNLI dataset, surpassing the previous SOTA models. This research demonstrates the potential of synthetic data augmentation in improving NLI models, offering a path forward for further advancements in natural language understanding tasks.

Related papers

• Generative adversarial networks vs large language models: a comparative study on synthetic tabular data generation [0.7373617024876725]
  We demonstrate the ability to generate high-language tabular data without task-specific fine-tuning or access to real-world data for pre-training. To benchmark GPT-4o, we compared the fidelity and privacy of LLM-generated synthetic data against data generated with the conditional generative adversarial network (CTGAN) Despite the zero-shot approach, GPT-4o outperformed CTGAN in preserving means, 95% confidence intervals, bivariate correlations, and data privacy of RWD, even at amplified sample sizes.
  arXiv  Detail & Related papers  (2025-02-20T12:56:16Z)
• Generating Realistic Tabular Data with Large Language Models [49.03536886067729]
  Large language models (LLM) have been used for diverse tasks, but do not capture the correct correlation between the features and the target variable. We propose a LLM-based method with three important improvements to correctly capture the ground-truth feature-class correlation in the real data. Our experiments show that our method significantly outperforms 10 SOTA baselines on 20 datasets in downstream tasks.
  arXiv  Detail & Related papers  (2024-10-29T04:14:32Z)
• Synthio: Augmenting Small-Scale Audio Classification Datasets with Synthetic Data [69.7174072745851]
  We present Synthio, a novel approach for augmenting small-scale audio classification datasets with synthetic data. To overcome the first challenge, we align the generations of the T2A model with the small-scale dataset using preference optimization. To address the second challenge, we propose a novel caption generation technique that leverages the reasoning capabilities of Large Language Models.
  arXiv  Detail & Related papers  (2024-10-02T22:05:36Z)
• Towards Effective and Efficient Continual Pre-training of Large Language Models [163.34610964970258]
  Continual pre-training (CPT) has been an important approach for adapting language models to specific domains or tasks. This paper presents a technical report for continually pre-training Llama-3 (8B) It significantly enhances the Chinese language ability and scientific reasoning ability of the backbone model.
  arXiv  Detail & Related papers  (2024-07-26T13:55:21Z)
• FLIGAN: Enhancing Federated Learning with Incomplete Data using GAN [1.5749416770494706]
  Federated Learning (FL) provides a privacy-preserving mechanism for distributed training of machine learning models on networked devices. We propose FLIGAN, a novel approach to address the issue of data incompleteness in FL. Our methodology adheres to FL's privacy requirements by generating synthetic data in a federated manner without sharing the actual data in the process.
  arXiv  Detail & Related papers  (2024-03-25T16:49:38Z)
• A synthetic data approach for domain generalization of NLI models [13.840374911669167]
  Natural Language Inference (NLI) remains an important benchmark task for LLMs. We show that synthetic high-quality datasets can adapt NLI models for zero-shot use in downstream applications. We show that models trained on this data have the best generalization to completely new downstream test settings.
  arXiv  Detail & Related papers  (2024-02-19T18:55:16Z)
• TarGEN: Targeted Data Generation with Large Language Models [51.87504111286201]
  TarGEN is a multi-step prompting strategy for generating high-quality synthetic datasets. We augment TarGEN with a method known as self-correction empowering LLMs to rectify inaccurately labeled instances. A comprehensive analysis of the synthetic dataset compared to the original dataset reveals similar or higher levels of dataset complexity and diversity.
  arXiv  Detail & Related papers  (2023-10-27T03:32:17Z)
• Let's Synthesize Step by Step: Iterative Dataset Synthesis with Large Language Models by Extrapolating Errors from Small Models [69.76066070227452]
  *Data Synthesis* is a promising way to train a small model with very little labeled data. We propose *Synthesis Step by Step* (**S3**), a data synthesis framework that shrinks this distribution gap. Our approach improves the performance of a small model by reducing the gap between the synthetic dataset and the real data.
  arXiv  Detail & Related papers  (2023-10-20T17:14:25Z)
• WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation [101.00109827301235]
  We introduce a novel paradigm for dataset creation based on human and machine collaboration. We use dataset cartography to automatically identify examples that demonstrate challenging reasoning patterns, and instruct GPT-3 to compose new examples with similar patterns. The resulting dataset, WANLI, consists of 108,357 natural language inference (NLI) examples that present unique empirical strengths.
  arXiv  Detail & Related papers  (2022-01-16T03:13:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.