Automatic Instruction Evolving for Large Language Models

• URL: http://arxiv.org/abs/2406.00770v1
• Date: Sun, 2 Jun 2024 15:09:00 GMT
• Title: Automatic Instruction Evolving for Large Language Models
• Authors: Weihao Zeng, Can Xu, Yingxiu Zhao, Jian-Guang Lou, Weizhu Chen,
• Abstract summary: Auto Evol-Instruct is an end-to-end framework that evolves instruction datasets using large language models without any human effort. Our experiments demonstrate that the best method optimized by Auto Evol-Instruct outperforms human-designed methods on various benchmarks.
• Score: 93.52437926313621
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Fine-tuning large pre-trained language models with Evol-Instruct has achieved encouraging results across a wide range of tasks. However, designing effective evolving methods for instruction evolution requires substantial human expertise. This paper proposes Auto Evol-Instruct, an end-to-end framework that evolves instruction datasets using large language models without any human effort. The framework automatically analyzes and summarizes suitable evolutionary strategies for the given instruction data and iteratively improves the evolving method based on issues exposed during the instruction evolution process. Our extensive experiments demonstrate that the best method optimized by Auto Evol-Instruct outperforms human-designed methods on various benchmarks, including MT-Bench, AlpacaEval, GSM8K, and HumanEval.

Related papers

• RAISE: Reinforenced Adaptive Instruction Selection For Large Language Models [48.63476198469349]
  We propose a task-objective-driven instruction selection framework RAISE. RAISE incorporates the entire instruction fine-tuning process into optimization. It selects instruction at each step based on the expected impact of instruction on model performance improvement.
  arXiv  Detail & Related papers  (2025-04-09T21:17:52Z)
• ToolACE-R: Tool Learning with Adaptive Self-Refinement [84.69651852838794]
  Tool learning allows Large Language Models to leverage external tools for solving complex user tasks. We propose ToolACE-R, a novel method that introduces adaptive self-refinement for tool invocations. Our results demonstrate the effectiveness of the proposed method, which is compatible with base models of various sizes.
  arXiv  Detail & Related papers  (2025-04-02T06:38:56Z)
• Instruction Tuning for Story Understanding and Generation with Weak Supervision [0.5530212768657544]
  We propose a novel approach called "Weak to Strong Instruction Tuning" for improving story generation. We show that our method significantly enhances performance in story comprehension and generation. Our work shows that adaptive instruction tuning can be a powerful tool in refining generative models for complex narrative tasks.
  arXiv  Detail & Related papers  (2025-01-26T15:59:31Z)
• Dynamic Skill Adaptation for Large Language Models [78.31322532135272]
  We present Dynamic Skill Adaptation (DSA), an adaptive and dynamic framework to adapt novel and complex skills to Large Language Models (LLMs) For every skill, we utilize LLMs to generate both textbook-like data which contains detailed descriptions of skills for pre-training and exercise-like data which targets at explicitly utilizing the skills to solve problems for instruction-tuning. Experiments on large language models such as LLAMA and Mistral demonstrate the effectiveness of our proposed methods in adapting math reasoning skills and social study skills.
  arXiv  Detail & Related papers  (2024-12-26T22:04:23Z)
• A Post-Training Enhanced Optimization Approach for Small Language Models [0.0]
  This paper proposes a continuous post-training alignment data construction method for small language models. The core of this method is based on the data guidance of large models, optimizing the diversity and accuracy of alignment data.
  arXiv  Detail & Related papers  (2024-11-05T09:32:26Z)
• Augmentation Policy Generation for Image Classification Using Large Language Models [3.038642416291856]
  We propose a strategy that uses large language models to automatically generate efficient augmentation policies. The proposed method was evaluated on medical imaging datasets, showing a clear improvement over state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-10-17T11:26:10Z)
• InternLM2 Technical Report [159.70692271378581]
  This paper introduces InternLM2, an open-source Large Language Models (LLMs) that outperforms its predecessors in comprehensive evaluations across 6 dimensions and 30 benchmarks. The pre-training process of InternLM2 is meticulously detailed, highlighting the preparation of diverse data types. InternLM2 efficiently captures long-term dependencies, initially trained on 4k tokens before advancing to 32k tokens in pre-training and fine-tuning stages.
  arXiv  Detail & Related papers  (2024-03-26T00:53:24Z)
• Learning to optimize by multi-gradient for multi-objective optimization [0.0]
  We introduce a new automatic learning paradigm for optimizing MOO problems, and propose a multi-gradient learning to optimize (ML2O) method. As a learning-based method, ML2O acquires knowledge of local landscapes by leveraging information from the current step. We show that our learned outperforms hand-designed competitors on training multi-task learning (MTL) neural network.
  arXiv  Detail & Related papers  (2023-11-01T14:55:54Z)
• InstOptima: Evolutionary Multi-objective Instruction Optimization via Large Language Model-based Instruction Operators [9.004528034920266]
  InstOptima treats instruction generation as an evolutionary multi-objective optimization problem. We introduce an objective-guided mechanism for operators, allowing the LLM to comprehend the objectives and enhance the quality of the generated instructions. Experimental results demonstrate improved fine-tuning performance and the generation of a diverse set of high-quality instructions.
  arXiv  Detail & Related papers  (2023-10-26T17:48:45Z)
• Improving Non-autoregressive Generation with Mixup Training [51.61038444990301]
  We present a non-autoregressive generation model based on pre-trained transformer models. We propose a simple and effective iterative training method called MIx Source and pseudo Target. Our experiments on three generation benchmarks including question generation, summarization and paraphrase generation, show that the proposed framework achieves the new state-of-the-art results.
  arXiv  Detail & Related papers  (2021-10-21T13:04:21Z)
• SDA: Improving Text Generation with Self Data Augmentation [88.24594090105899]
  We propose to improve the standard maximum likelihood estimation (MLE) paradigm by incorporating a self-imitation-learning phase for automatic data augmentation. Unlike most existing sentence-level augmentation strategies, our method is more general and could be easily adapted to any MLE-based training procedure.
  arXiv  Detail & Related papers  (2021-01-02T01:15:57Z)
• Selecting Informative Contexts Improves Language Model Finetuning [66.26521454263343]
  We present a general fine-tuning method that we call information gain filtration. During fine-tuning, a secondary learner selects informative examples and skips uninformative ones. We show that our method has consistent improvement across datasets, fine-tuning tasks, and language model architectures.
  arXiv  Detail & Related papers  (2020-05-01T02:01:18Z)
• Recall and Learn: Fine-tuning Deep Pretrained Language Models with Less Forgetting [66.45372974713189]
  We propose a recall and learn mechanism, which adopts the idea of multi-task learning and jointly learns pretraining tasks and downstream tasks. Experiments show that our method achieves state-of-the-art performance on the GLUE benchmark. We provide open-source RecAdam, which integrates the proposed mechanisms into Adam to facility the NLP community.
  arXiv  Detail & Related papers  (2020-04-27T08:59:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.