Many-Tier Instruction Hierarchy in LLM Agents

• URL: http://arxiv.org/abs/2604.09443v3
• Date: Tue, 14 Apr 2026 15:04:47 GMT
• Title: Many-Tier Instruction Hierarchy in LLM Agents
• Abstract summary: Many-Tier Instruction Hierarchy (ManyIH) is a paradigm for resolving instruction conflicts among instructions with arbitrarily many privilege levels.<n>ManyIH-Bench composes constraints developed by LLMs and verified by humans to create realistic and difficult test cases spanning 46 real-world agents.
• Score: 71.50171548872596
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language model agents receive instructions from many sources-system messages, user prompts, tool outputs, other agents, and more-each carrying different levels of trust and authority. When these instructions conflict, agents must reliably follow the highest-privilege instruction to remain safe and effective. The dominant paradigm, instruction hierarchy (IH), assumes a fixed, small set of privilege levels (typically fewer than five) defined by rigid role labels (e.g., system > user). This is inadequate for real-world agentic settings, where conflicts can arise across far more sources and contexts. In this work, we propose Many-Tier Instruction Hierarchy (ManyIH), a paradigm for resolving instruction conflicts among instructions with arbitrarily many privilege levels. We introduce ManyIH-Bench, the first benchmark for ManyIH. ManyIH-Bench requires models to navigate up to 12 levels of conflicting instructions with varying privileges, comprising 853 agentic tasks (427 coding and 426 instruction-following). ManyIH-Bench composes constraints developed by LLMs and verified by humans to create realistic and difficult test cases spanning 46 real-world agents. Our experiments show that even the current frontier models perform poorly (~40% accuracy) when instruction conflict scales. This work underscores the urgent need for methods that explicitly target fine-grained, scalable instruction conflict resolution in agentic settings.

Related papers

• MTAC-IFBench: Benchmarking Instruction-Following in Multi-Turn Agentic Coding [83.0805548642958]
  MTAC-IFBench is a comprehensive benchmark for instruction-following in software development.<n>It features multi-turn progressive software development instructions with diverse constraints spanning 6 primary and 18 secondary categories.<n>MTAC-IFBench identifies significant deficiencies in existing code agents in multi-turn instruction-following, with their performance degrading rapidly as the interaction session grows longer.
  arXiv  Detail & Related papers  (2026-09-14T03:57:37Z)
• IH-Benchmark: A Conflict-Centered Benchmark for Instruction-Hierarchy Robustness in LLM Applications [1.9966419762834917]
  IH-Benchmark is a conflict-centered benchmark for instruction-hierarchy robustness across direct system-user conflicts (S>U) and tool-mediated user-tool (U>T) conflicts.<n>Across 37 evaluated models, hierarchy compliance ranges from 98.2% to 20.5%.<n>We find that strong S>U compliance is not a reliable proxy for U>T robustness.
  arXiv  Detail & Related papers  (2026-07-28T17:06:13Z)
• Hierarchical Alignment: Enforcing Hierarchical Instruction-Following in LLMs through Logical Consistency [21.166652815597782]
  We propose Neuro-Symbolic Hierarchical Alignment (NSHA) for hierarchical instruction-following by explicitly modeling and enforcing instruction priorities.<n>We evaluate our approach on rule following, task execution, tool use, and safety, covering both single-turn and multi-turn interactions.
  arXiv  Detail & Related papers  (2026-04-10T07:59:11Z)
• Ask or Assume? Uncertainty-Aware Clarification-Seeking in Coding Agents [4.301199871195023]
  Large Language Model (LLM) agents are increasingly deployed in open-ended domains like software engineering.<n>We evaluate the clarification-seeking abilities of LLM agents on an underspecified variant of SWE-bench Verified.<n>We propose an uncertainty-aware multi-agent scaffold that explicitly decouples underspecification detection from code execution.
  arXiv  Detail & Related papers  (2026-03-27T09:56:26Z)
• Collaborative Multi-Agent Test-Time Reinforcement Learning for Reasoning [112.16686518063456]
  We introduce textbfMulti-Agent Test-Time Reinforcement Learning (MATTRL), a framework that injects structured textual experience into multi-agent deliberation at inference time.<n>MATTRL forms a multi-expert team of specialists for multi-turn discussions, retrieves and integrates test-time experiences, and reaches consensus for final decision-making.<n>Across challenging benchmarks in medicine, math, and education, MATTRL improves accuracy by an average of 3.67% over a multi-agent baseline, and by 8.67% over comparable single-agent baselines
  arXiv  Detail & Related papers  (2026-01-14T17:57:43Z)
• EDIT-Bench: Evaluating LLM Abilities to Perform Real-World Instructed Code Edits [72.23150343093447]
  We introduce EDIT-Bench, a benchmark for evaluatingstructed code editing capabilities grounded in real-world usage.<n>EDIT-Bench comprises of 545 problems, multiple natural and programming languages, and a diverse set of real-world use cases.<n>We find that model performance varies across different categories of user instructions.
  arXiv  Detail & Related papers  (2025-11-06T16:05:28Z)
• Reasoning Up the Instruction Ladder for Controllable Language Models [26.068755167791505]
  Large language model (LLM) based systems take on high-stakes roles in real-world decision-making.<n>Enforcement of an instruction hierarchy (IH) in LLMs is critical for the reliability and controllability of LLMs.<n>In this work, we reframe instruction hierarchy resolution as a reasoning task.
  arXiv  Detail & Related papers  (2025-10-30T22:13:31Z)
• Diagnose, Localize, Align: A Full-Stack Framework for Reliable LLM Multi-Agent Systems under Instruction Conflicts [75.20929587906228]
  Large Language Model (LLM)-powered multi-agent systems (MAS) have rapidly advanced collaborative reasoning, tool use, and role-specialized coordination in complex tasks.<n>However, reliability-critical deployment remains hindered by a systemic failure mode: hierarchical compliance under instruction conflicts.
  arXiv  Detail & Related papers  (2025-09-27T08:43:34Z)
• Can Language Models Follow Multiple Turns of Entangled Instructions? [109.4355301539557]
  Real-world scenarios often require consistency across multiple instructions over time.<n>This work presents a systematic investigation of large language models' capabilities in handling multiple turns of instructions.<n>We construct MultiTurnInstructwith $sim$1.1K high-quality multi-turn conversations through the human-in-the-loop approach.
  arXiv  Detail & Related papers  (2025-03-17T14:31:37Z)
• Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchy [53.54777131440989]
  Large Language Models (LLMs) are susceptible to security and safety threats.<n>One major cause of these vulnerabilities is the lack of an instruction hierarchy.<n>We introduce the instructional segment Embedding (ISE) technique, inspired by BERT, to modern large language models.
  arXiv  Detail & Related papers  (2024-10-09T12:52:41Z)
• The SIFo Benchmark: Investigating the Sequential Instruction Following Ability of Large Language Models [48.455388608863785]
  We introduce a benchmark designed to evaluate models' abilities to follow multiple instructions through sequential instruction following tasks. Our benchmark evaluates instruction following using four tasks (text modification, question answering, mathematics, and security rules) More recent and larger models significantly outperform their older and smaller counterparts on the SIFo tasks, validating the benchmark's effectiveness.
  arXiv  Detail & Related papers  (2024-06-28T15:34:26Z)
• Towards Unified Task Embeddings Across Multiple Models: Bridging the Gap for Prompt-Based Large Language Models and Beyond [16.913115978881866]
  We propose a framework for unified task embeddings (FUTE), task embeddings from various models, including smaller language models and Large Language Models with varied prompts, within a single vector space. Such uniformity enables comparison and analysis of similarities amongst different models, broadening the scope and utility of existing task embedding methods in multi-model scenarios.
  arXiv  Detail & Related papers  (2024-02-22T13:13:31Z)
• Room Clearance with Feudal Hierarchical Reinforcement Learning [2.867517731896504]
  We introduce a new simulation environment, "it", designed as a tool to build scenarios that can drive RL research in a direction useful for military analysis. We focus on an abstracted and simplified room clearance scenario, where a team of blue agents have to make their way through a building and ensure that all rooms are cleared of enemy red agents. We implement a multi-agent version of feudal hierarchical RL that introduces a command hierarchy where a commander at the higher level sends orders to multiple agents at the lower level who simply have to learn to follow these orders. We find that breaking the task down in this way allows us to
  arXiv  Detail & Related papers  (2021-05-24T15:05:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.