Conifer: Improving Complex Constrained Instruction-Following Ability of Large Language Models

• URL: http://arxiv.org/abs/2404.02823v1
• Date: Wed, 3 Apr 2024 15:55:39 GMT
• Title: Conifer: Improving Complex Constrained Instruction-Following Ability of Large Language Models
• Authors: Haoran Sun, Lixin Liu, Junjie Li, Fengyu Wang, Baohua Dong, Ran Lin, Ruohui Huang,
• Abstract summary: We introduce Conifer, an instruction tuning dataset for large language models. We train models with Conifer to follow instructions with complex constraints. On several instruction-following benchmarks, our 7B model outperforms the state-of-the-art open-source 7B models.
• Score: 23.17547206140014
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The ability of large language models (LLMs) to follow instructions is crucial to real-world applications. Despite recent advances, several studies have highlighted that LLMs struggle when faced with challenging instructions, especially those that include complex constraints, hindering their effectiveness in various tasks. To address this challenge, we introduce Conifer, a novel instruction tuning dataset, designed to enhance LLMs to follow multi-level instructions with complex constraints. Utilizing GPT-4, we curate the dataset by a series of LLM-driven refinement processes to ensure high quality. We also propose a progressive learning scheme that emphasizes an easy-to-hard progression, and learning from process feedback. Models trained with Conifer exhibit remarkable improvements in instruction-following abilities, especially for instructions with complex constraints. On several instruction-following benchmarks, our 7B model outperforms the state-of-the-art open-source 7B models, even exceeds the performance of models 10 times larger on certain metrics. All the code and Conifer dataset are available at https://www.github.com/ConiferLM/Conifer.

Related papers

• MuSC: Improving Complex Instruction Following with Multi-granularity Self-Contrastive Training [36.483136685734735]
  We propose a Multi-granularity Self-Contrastive Training (MuSC) framework to improve the complex instruction alignment without relying on a stronger model. Our method is evaluated on open-sourced models, and experiment results show our method achieves significant improvement on both complex and general instruction-following benchmarks.
  arXiv  Detail & Related papers  (2025-02-17T08:12:49Z)
• The Inherent Limits of Pretrained LLMs: The Unexpected Convergence of Instruction Tuning and In-Context Learning Capabilities [51.594836904623534]
  We investigate whether instruction-tuned models possess fundamentally different capabilities from base models that are prompted using in-context examples. We show that the performance of instruction-tuned models is significantly correlated with the in-context performance of their base counterparts. Specifically, we extend this understanding to instruction-tuned models, suggesting that their pretraining data similarly sets a limiting boundary on the tasks they can solve.
  arXiv  Detail & Related papers  (2025-01-15T10:57:55Z)
• WarriorCoder: Learning from Expert Battles to Augment Code Large Language Models [67.15146980023621]
  We propose WarriorCoder, a novel paradigm learns from expert battles to address limitations of current approaches. We create an arena where leading expert code LLMs challenge each other, with evaluations conducted by impartial judges. This competitive framework generates novel training data from scratch, leveraging the strengths of all participants.
  arXiv  Detail & Related papers  (2024-12-23T08:47:42Z)
• Constraint Back-translation Improves Complex Instruction Following of Large Language Models [55.60192044049083]
  Large language models (LLMs) struggle to follow instructions with complex constraints in format, length, etc. Previous works conduct post-training on complex instruction-response pairs generated by feeding complex instructions to advanced LLMs. We propose a novel data generation technique, constraint back-translation.
  arXiv  Detail & Related papers  (2024-10-31T17:42:26Z)
• Evolutionary Contrastive Distillation for Language Model Alignment [35.94171633370035]
  Evolutionary Contrastive Distillation (ECD) is a novel method for generating high-quality synthetic preference data. Our method yields a 7B model that exceeds the complex instruction-following performance of current SOTA 7B models.
  arXiv  Detail & Related papers  (2024-10-10T01:04:03Z)
• Benchmarking Complex Instruction-Following with Multiple Constraints Composition [72.82640456309821]
  How to evaluate the ability of complex instruction-following of large language models (LLMs) has become a critical research problem. Existing benchmarks mainly focus on modeling different types of constraints in human instructions while neglecting the composition of different constraints. We propose ComplexBench, a benchmark for comprehensively evaluating the ability of LLMs to follow complex instructions composed of multiple constraints.
  arXiv  Detail & Related papers  (2024-07-04T14:50:45Z)
• Learning to Plan for Retrieval-Augmented Large Language Models from Knowledge Graphs [59.76268575344119]
  We introduce a novel framework for enhancing large language models' (LLMs) planning capabilities by using planning data derived from knowledge graphs (KGs) LLMs fine-tuned with KG data have improved planning capabilities, better equipping them to handle complex QA tasks that involve retrieval.
  arXiv  Detail & Related papers  (2024-06-20T13:07:38Z)
• From Complex to Simple: Enhancing Multi-Constraint Complex Instruction Following Ability of Large Language Models [43.869374263102934]
  We study what training data is effective in enhancing complex constraints following abilities. We find that training LLMs with instructions containing multiple constraints enhances their understanding of complex instructions. Our methods improve models' ability to follow instructions generally and generalize effectively across out-of-domain, in-domain, and adversarial settings.
  arXiv  Detail & Related papers  (2024-04-24T12:51:14Z)
• Can Large Language Models Understand Real-World Complex Instructions? [54.86632921036983]
  Large language models (LLMs) can understand human instructions, but struggle with complex instructions. Existing benchmarks are insufficient to assess LLMs' ability to understand complex instructions. We propose CELLO, a benchmark for evaluating LLMs' ability to follow complex instructions systematically.
  arXiv  Detail & Related papers  (2023-09-17T04:18:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.