IHEval: Evaluating Language Models on Following the Instruction Hierarchy

• URL: http://arxiv.org/abs/2502.08745v1
• Date: Wed, 12 Feb 2025 19:35:28 GMT
• Title: IHEval: Evaluating Language Models on Following the Instruction Hierarchy
• Authors: Zhihan Zhang, Shiyang Li, Zixuan Zhang, Xin Liu, Haoming Jiang, Xianfeng Tang, Yifan Gao, Zheng Li, Haodong Wang, Zhaoxuan Tan, Yichuan Li, Qingyu Yin, Bing Yin, Meng Jiang,
• Abstract summary: The instruction hierarchy establishes a priority order from system messages to user messages, conversation history, and tool outputs. Despite its importance, this topic receives limited attention, and there is a lack of comprehensive benchmarks for evaluating models' ability to follow the instruction hierarchy. We bridge this gap by introducing IHEval, a novel benchmark covering cases where instructions in different priorities either align or conflict.
• Score: 67.33509094445104
• License:
• Abstract: The instruction hierarchy, which establishes a priority order from system messages to user messages, conversation history, and tool outputs, is essential for ensuring consistent and safe behavior in language models (LMs). Despite its importance, this topic receives limited attention, and there is a lack of comprehensive benchmarks for evaluating models' ability to follow the instruction hierarchy. We bridge this gap by introducing IHEval, a novel benchmark comprising 3,538 examples across nine tasks, covering cases where instructions in different priorities either align or conflict. Our evaluation of popular LMs highlights their struggle to recognize instruction priorities. All evaluated models experience a sharp performance decline when facing conflicting instructions, compared to their original instruction-following performance. Moreover, the most competitive open-source model only achieves 48% accuracy in resolving such conflicts. Our results underscore the need for targeted optimization in the future development of LMs.

Related papers

• Natural Language Inference Improves Compositionality in Vision-Language Models [35.71815423077561]
  We present a principled approach that generates entailments and contradictions from a given premise. CECE produces lexically diverse sentences while maintaining their core meaning. We achieve significant improvements over previous methods without requiring additional fine-tuning.
  arXiv  Detail & Related papers  (2024-10-29T17:54:17Z)
• Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchy [53.54777131440989]
  Large Language Models (LLMs) are susceptible to security and safety threats. One major cause of these vulnerabilities is the lack of an instruction hierarchy. We introduce the instructional segment Embedding (ISE) technique, inspired by BERT, to modern large language models.
  arXiv  Detail & Related papers  (2024-10-09T12:52:41Z)
• The SIFo Benchmark: Investigating the Sequential Instruction Following Ability of Large Language Models [48.455388608863785]
  We introduce a benchmark designed to evaluate models' abilities to follow multiple instructions through sequential instruction following tasks. Our benchmark evaluates instruction following using four tasks (text modification, question answering, mathematics, and security rules) More recent and larger models significantly outperform their older and smaller counterparts on the SIFo tasks, validating the benchmark's effectiveness.
  arXiv  Detail & Related papers  (2024-06-28T15:34:26Z)
• Evaluating Generative Language Models in Information Extraction as Subjective Question Correction [49.729908337372436]
  We propose a new evaluation method, SQC-Score. Inspired by the principles in subjective question correction, we propose a new evaluation method, SQC-Score. Results on three information extraction tasks show that SQC-Score is more preferred by human annotators than the baseline metrics.
  arXiv  Detail & Related papers  (2024-04-04T15:36:53Z)
• FollowEval: A Multi-Dimensional Benchmark for Assessing the Instruction-Following Capability of Large Language Models [42.72420855478716]
  FollowEval benchmark is composed of instances in both English and Chinese. Each test example is designed to evaluate more than one dimension. We have evaluated various LLMs using the FollowEval benchmark and found that their performance significantly lags behind that of humans.
  arXiv  Detail & Related papers  (2023-11-16T11:53:31Z)
• Don't Make Your LLM an Evaluation Benchmark Cheater [142.24553056600627]
  Large language models(LLMs) have greatly advanced the frontiers of artificial intelligence, attaining remarkable improvement in model capacity. To assess the model performance, a typical approach is to construct evaluation benchmarks for measuring the ability level of LLMs. We discuss the potential risk and impact of inappropriately using evaluation benchmarks and misleadingly interpreting the evaluation results.
  arXiv  Detail & Related papers  (2023-11-03T14:59:54Z)
• Investigating the Efficacy of Large Language Models in Reflective Assessment Methods through Chain of Thoughts Prompting [0.2552922646705803]
  Chain of Thought(CoT) prompting method has been proposed as a means to enhance LLMs' proficiency in complex reasoning tasks. The primary aim of this research is to assess how well four language models can grade reflective essays of third-year medical students.
  arXiv  Detail & Related papers  (2023-09-30T06:25:27Z)
• FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets [69.91340332545094]
  We introduce FLASK, a fine-grained evaluation protocol for both human-based and model-based evaluation. We experimentally observe that the fine-graininess of evaluation is crucial for attaining a holistic view of model performance.
  arXiv  Detail & Related papers  (2023-07-20T14:56:35Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.