Leveraging Web-Crawled Data for High-Quality Fine-Tuning

• URL: http://arxiv.org/abs/2408.08003v1
• Date: Thu, 15 Aug 2024 08:12:52 GMT
• Title: Leveraging Web-Crawled Data for High-Quality Fine-Tuning
• Authors: Jing Zhou, Chenglin Jiang, Wei Shen, Xiao Zhou, Xiaonan He,
• Abstract summary: We argue that web-crawled data can still serve as a valuable source for high-quality supervised fine-tuning without relying on advanced models like GPT-4. We create a paired training dataset automatically by aligning web-crawled data with a smaller set of high-quality data. Our experiments show that training with the model-transformed data yields better results, surpassing training with only high-quality data by an average score of 9.4% in Chinese math problems.
• Score: 24.19939701706869
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Most large language models are fine-tuned using either expensive human-annotated data or GPT-4 generated data which cannot guarantee performance in certain domains. We argue that although the web-crawled data often has formatting errors causing semantic inaccuracies, it can still serve as a valuable source for high-quality supervised fine-tuning in specific domains without relying on advanced models like GPT-4. To this end, we create a paired training dataset automatically by aligning web-crawled data with a smaller set of high-quality data. By training a language model on this dataset, we can convert web data with irregular formats into high-quality ones. Our experiments show that training with the model-transformed data yields better results, surpassing training with only high-quality data by an average score of 9.4% in Chinese math problems. Additionally, our 7B model outperforms several open-source models larger than 32B and surpasses well-known closed-source models such as GPT-3.5, highlighting the efficacy of our approach.

Related papers

• Little Giants: Synthesizing High-Quality Embedding Data at Scale [71.352883755806]
  We introduce SPEED, a framework that aligns open-source small models to efficiently generate large-scale embedding data. SPEED uses only less than 1/10 of the GPT API calls, outperforming the state-of-the-art embedding model E5_mistral when both are trained solely on their synthetic data.
  arXiv  Detail & Related papers  (2024-10-24T10:47:30Z)
• Forewarned is Forearmed: Leveraging LLMs for Data Synthesis through Failure-Inducing Exploration [90.41908331897639]
  Large language models (LLMs) have significantly benefited from training on diverse, high-quality task-specific data. We present a novel approach, ReverseGen, designed to automatically generate effective training samples.
  arXiv  Detail & Related papers  (2024-10-22T06:43:28Z)
• Towards Effective and Efficient Continual Pre-training of Large Language Models [163.34610964970258]
  Continual pre-training (CPT) has been an important approach for adapting language models to specific domains or tasks. This paper presents a technical report for continually pre-training Llama-3 (8B) It significantly enhances the Chinese language ability and scientific reasoning ability of the backbone model.
  arXiv  Detail & Related papers  (2024-07-26T13:55:21Z)
• AgentInstruct: Toward Generative Teaching with Agentic Flows [12.192372792525726]
  We focus on using synthetic data for post-training, specifically creating data by powerful models to teach a new skill or behavior to another model. We introduce AgentInstruct, an agentic framework for automatically creating large amounts of diverse and high-quality synthetic data. We demonstrate the utility of AgentInstruct by creating a post training dataset of 25M pairs to teach language models different skills, such as text editing, creative writing, tool usage, coding, reading comprehension, etc.
  arXiv  Detail & Related papers  (2024-07-03T21:01:12Z)
• Smaller Language Models are capable of selecting Instruction-Tuning Training Data for Larger Language Models [39.65879784788677]
  We introduce a novel training data selection based on the learning percentage of the samples. We assert that current language models possess the capability to autonomously select high-quality training data. Our paper introduces a novel approach to training data selection, showcasing a more efficient alternative.
  arXiv  Detail & Related papers  (2024-02-16T03:39:37Z)
• Efficient Grammatical Error Correction Via Multi-Task Training and Optimized Training Schedule [55.08778142798106]
  We propose auxiliary tasks that exploit the alignment between the original and corrected sentences. We formulate each task as a sequence-to-sequence problem and perform multi-task training. We find that the order of datasets used for training and even individual instances within a dataset may have important effects on the final performance.
  arXiv  Detail & Related papers  (2023-11-20T14:50:12Z)
• On the Impact of Cross-Domain Data on German Language Models [20.758967185444416]
  We present a German dataset comprising texts from five domains, along with another dataset aimed at containing high-quality data. Through training a series of models ranging between 122M and 750M parameters on both datasets, we conduct a comprehensive benchmark on multiple downstream tasks. Our findings demonstrate that the models trained on the cross-domain dataset outperform those trained on quality data alone, leading to improvements up to $4.45%$ over the previous state-of-the-art.
  arXiv  Detail & Related papers  (2023-10-11T09:09:55Z)
• WanJuan: A Comprehensive Multimodal Dataset for Advancing English and Chinese Large Models [69.96148259273065]
  "Wan Juan" is a large-scale multimodal dataset composed of both Chinese and English data, collected from a wide range of web sources. It was utilized in the training of InternLM, a model that demonstrated significant advantages in multi-dimensional evaluations when compared to models of a similar scale.
  arXiv  Detail & Related papers  (2023-08-21T14:40:48Z)
• RLBoost: Boosting Supervised Models using Deep Reinforcement Learning [0.0]
  We present RLBoost, an algorithm that uses deep reinforcement learning strategies to evaluate a particular dataset and obtain a model capable of estimating the quality of any new data. The results of the article show that this model obtains better and more stable results than other state-of-the-art algorithms such as LOO, DataShapley or DVRL.
  arXiv  Detail & Related papers  (2023-05-23T14:38:33Z)
• A Data-centric Framework for Improving Domain-specific Machine Reading Comprehension Datasets [5.673449249014538]
  Low-quality data can cause downstream problems in high-stakes applications. Data-centric approach emphasizes on improving dataset quality to enhance model performance.
  arXiv  Detail & Related papers  (2023-04-02T08:26:38Z)
• Teacher Guided Training: An Efficient Framework for Knowledge Transfer [86.6784627427194]
  We propose the teacher-guided training (TGT) framework for training a high-quality compact model. TGT exploits the fact that the teacher has acquired a good representation of the underlying data domain. We find that TGT can improve accuracy on several image classification benchmarks and a range of text classification and retrieval tasks.
  arXiv  Detail & Related papers  (2022-08-14T10:33:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.