SHED: Shapley-Based Automated Dataset Refinement for Instruction Fine-Tuning

• URL: http://arxiv.org/abs/2405.00705v2
• Date: Wed, 30 Oct 2024 06:33:02 GMT
• Title: SHED: Shapley-Based Automated Dataset Refinement for Instruction Fine-Tuning
• Authors: Yexiao He, Ziyao Wang, Zheyu Shen, Guoheng Sun, Yucong Dai, Yongkai Wu, Hongyi Wang, Ang Li,
• Abstract summary: Large Language Models can achieve desirable performance with only a small amount of high-quality data. Identifying high-quality data from vast datasets to curate small yet effective datasets has emerged as a critical challenge. We introduce SHED, an automated dataset refinement framework based on Shapley value for instruction fine-tuning.
• Score: 16.307467144690683
• License:
• Abstract: The pre-trained Large Language Models (LLMs) can be adapted for many downstream tasks and tailored to align with human preferences through fine-tuning. Recent studies have discovered that LLMs can achieve desirable performance with only a small amount of high-quality data, suggesting that a large amount of the data in these extensive datasets is redundant or even harmful. Identifying high-quality data from vast datasets to curate small yet effective datasets has emerged as a critical challenge. In this paper, we introduce SHED, an automated dataset refinement framework based on Shapley value for instruction fine-tuning. SHED eliminates the need for human intervention or the use of commercial LLMs. Moreover, the datasets curated through SHED exhibit transferability, indicating they can be reused across different LLMs with consistently high performance. We conduct extensive experiments to evaluate the datasets curated by SHED. The results demonstrate SHED's superiority over state-of-the-art methods across various tasks and LLMs; notably, datasets comprising only 10% of the original data selected by SHED achieve performance comparable to or surpassing that of the full datasets.

Related papers

• Star-Agents: Automatic Data Optimization with LLM Agents for Instruction Tuning [71.2981957820888]
  We propose a novel Star-Agents framework, which automates the enhancement of data quality across datasets. The framework initially generates diverse instruction data with multiple LLM agents through a bespoke sampling method. The generated data undergo a rigorous evaluation using a dual-model method that assesses both difficulty and quality.
  arXiv  Detail & Related papers  (2024-11-21T02:30:53Z)
• Not All LLM-Generated Data Are Equal: Rethinking Data Weighting in Text Classification [7.357494019212501]
  We propose efficient weighted-loss approaches to align synthetic data with real-world distribution. We empirically assessed the effectiveness of our method on multiple text classification tasks.
  arXiv  Detail & Related papers  (2024-10-28T20:53:49Z)
• A CLIP-Powered Framework for Robust and Generalizable Data Selection [51.46695086779598]
  Real-world datasets often contain redundant and noisy data, imposing a negative impact on training efficiency and model performance. Data selection has shown promise in identifying the most representative samples from the entire dataset. We propose a novel CLIP-powered data selection framework that leverages multimodal information for more robust and generalizable sample selection.
  arXiv  Detail & Related papers  (2024-10-15T03:00:58Z)
• Data Advisor: Dynamic Data Curation for Safety Alignment of Large Language Models [79.65071553905021]
  We propose Data Advisor, a method for generating data that takes into account the characteristics of the desired dataset. Data Advisor monitors the status of the generated data, identifies weaknesses in the current dataset, and advises the next iteration of data generation.
  arXiv  Detail & Related papers  (2024-10-07T17:59:58Z)
• Empirical Insights on Fine-Tuning Large Language Models for Question-Answering [50.12622877002846]
  Large language models (LLMs) encode extensive world knowledge through pre-training on massive datasets, which can be fine-tuned for the question-answering (QA) task. We categorize supervised fine-tuning (SFT) data based on the extent of knowledge memorized by the pretrained LLMs. Our experiments show that as few as 60 data points during the SFT stage can activate the knowledge encoded during pre-training, enabling LLMs to perform the QA task.
  arXiv  Detail & Related papers  (2024-09-24T07:38:38Z)
• Efficacy of Synthetic Data as a Benchmark [3.2968976262860408]
  We investigate the effectiveness of generating synthetic data through large language models (LLMs) Our experiments show that while synthetic data can effectively capture performance of various methods for simpler tasks, it falls short for more complex tasks like named entity recognition. We propose a new metric called the bias factor, which evaluates the biases introduced when the same LLM is used to both generate benchmarking data and to perform the tasks.
  arXiv  Detail & Related papers  (2024-09-18T13:20:23Z)
• LESS: Selecting Influential Data for Targeted Instruction Tuning [64.78894228923619]
  We propose LESS, an efficient algorithm to estimate data influences and perform Low-rank gradiEnt Similarity Search for instruction data selection. We show that training on a LESS-selected 5% of the data can often outperform training on the full dataset across diverse downstream tasks. Our method goes beyond surface form cues to identify data that the necessary reasoning skills for the intended downstream application.
  arXiv  Detail & Related papers  (2024-02-06T19:18:04Z)
• Curated LLM: Synergy of LLMs and Data Curation for tabular augmentation in low-data regimes [57.62036621319563]
  We introduce CLLM, which leverages the prior knowledge of Large Language Models (LLMs) for data augmentation in the low-data regime. We demonstrate the superior performance of CLLM in the low-data regime compared to conventional generators.
  arXiv  Detail & Related papers  (2023-12-19T12:34:46Z)
• Rethinking the Instruction Quality: LIFT is What You Need [20.829372251475476]
  Existing quality improvement methods alter instruction data through dataset expansion or curation. We propose LIFT (LLM Instruction Fusion Transfer), a novel and versatile paradigm designed to elevate the instruction quality to new heights. Experimental results demonstrate that, even with a limited quantity of high-quality instruction data selected by our paradigm, LLMs consistently uphold robust performance across various tasks.
  arXiv  Detail & Related papers  (2023-12-12T03:30:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.