Instruction Complexity Induces Positional Collapse in Adversarial LLM Evaluation

• URL: http://arxiv.org/abs/2604.27249v1
• Date: Wed, 29 Apr 2026 22:48:24 GMT
• Title: Instruction Complexity Induces Positional Collapse in Adversarial LLM Evaluation
• Abstract summary: Vague adversarial instructions produce moderate accuracy reduction with preserved content engagement.<n>A two-step answer-aware avoidance instruction produces extreme positional collapse with near-total concentration on a single response position.<n>Results suggest that instruction complexity can determine whether adversarial compliance uses content-aware or content-blind mechanisms.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When instructed to underperform on multiple-choice evaluations, do language models engage with question content or fall back on positional shortcuts? We map the boundary between these regimes using a six-condition adversarial instruction-specificity gradient administered to two instruction-tuned LLMs (Llama-3-8B and Llama-3.1-8B) on 2,000 MMLU-Pro items. Distributional screening (response-position entropy) and an independent content-engagement criterion (difficulty-accuracy correlation) jointly characterise each condition. The gradient reveals three regimes rather than a monotonic transition. Vague adversarial instructions produce moderate accuracy reduction with preserved content engagement. Standard sandbagging and capability-imitation instructions produce positional entropy collapse with partial content engagement. A two-step answer-aware avoidance instruction produces extreme positional collapse, with near-total concentration on a single response position (99.9% and 87.4%) and no measurable content sensitivity. This was the only multi-step instruction tested, and it produced the most extreme shortcut. The attractor position matches each model's content-absent null-prompt default. The effect replicates across both models and four academic domains. Distributional collapse and content engagement can co-occur (50% concordance between screening criteria), indicating that entropy-based screening and difficulty-based content assessment capture partially independent dimensions of response validity. Results suggest that instruction complexity can determine whether adversarial compliance uses content-aware or content-blind mechanisms in small instruction-tuned LLMs under greedy decoding.

Related papers

• SEER: A Self-Grounded Evidence Interface for Controlled Spatial Relation Classification [71.9783246097687]
  SEER hides candidate relations during pair localization, constructs a query-specific view with explicit subject/object roles, and retains the full image and sparse box geometry as complementary evidence.<n>For relation-choice protocols with exact inverse support, an optional refinement swaps the entity roles and changes the forward decision only when exactly one visual state obeys the corresponding inverse relation.<n>The unchanged protocol yields +4.35 to +11.79 on all 2,434 filtered EmbSpatial pair-relation questions across three models.
  arXiv  Detail & Related papers  (2026-08-04T13:16:15Z)
• Evaluating LLM Personalization via Semantic Constraint Verification [25.539904066738288]
  We introduce Natural Language Inference Constraint Verification (NLICV)<n>NLICV maps sentence meanings to truth-condition sets to verify personalization constraints via a Natural Language Inference (NLI) model.<n>Experiments demonstrate that NLICV aligns closely with human annotations while drastically reducing the latency and token costs associated with LLM judges.
  arXiv  Detail & Related papers  (2026-06-15T08:04:56Z)
• R-APS: Compositional Reasoning and In-Context Meta-Learning for Constrained Design via Reflective Adversarial Pareto Search [18.8323743697237]
  Large language models (LLMs) are fluent on open-ended tasks, yet in agentic settings, fluency does not ensure reliable delivery.<n>We trace this gap to three coupled structural failures: errors propagate without localization, worst-case perturbations go unevaluated, and accumulated knowledge is never invalidated.<n>We argue these share a root cause: abductive, counterfactual, meta-inductive, corrective, and inductive reasoning pull a shared context in incompatible directions.
  arXiv  Detail & Related papers  (2026-06-03T12:45:39Z)
• On the Limits of LLM Adaptability: Impact of Model-Internalized Priors on Annotation Task Performance [0.7303825588015752]
  We investigate how model-internalized priors interact with user-provided instructions.<n>We find that nearly two-thirds of zero-shot errors are resistant to correction.<n>High-confidence errors prove especially resistant to correction.
  arXiv  Detail & Related papers  (2026-05-30T01:21:14Z)
• Revisiting Uncertainty: On Evidential Learning for Partially Relevant Video Retrieval [70.47320120707029]
  Partially relevant video retrieval aims to retrieve untrimmed videos using text queries that describe only partial content.<n>In this setting, vague queries often induce semantic ambiguity across videos, a challenge that is further exacerbated by the sparse temporal supervision within videos.<n>We propose Holmes, a hierarchical evidential learning framework that aggregates multi-granular cross-modal evidence to quantify and model uncertainty explicitly.
  arXiv  Detail & Related papers  (2026-05-07T12:06:13Z)
• Paraphrase-Induced Output-Mode Collapse: When LLMs Break Character Under Semantically Equivalent Inputs [0.9525172018746524]
  We observe a systematic failure mode we call prompt-variant output-mode collapse.<n>When a closed-form prompt asks for a bare label or a single choice token, content-preserving prompt variants can push the model into conversational prose.<n>We release PARACONSIST, a benchmark of 150 base queries with five lexical, syntactic, and semantic-expansion prompt variants each.
  arXiv  Detail & Related papers  (2026-05-06T09:11:10Z)
• Compiling Deterministic Structure into SLM Harnesses [0.6117371161379209]
  Small language models (SLMs) are costly and face data sovereignty limits for high-volume use.<n>We propose Semantic Gradient Descent (SGDe), a teacher-student framework that compiles agentic improvement into discrete execution plans.
  arXiv  Detail & Related papers  (2026-04-19T14:04:29Z)
• Sparse Visual Thought Circuits in Vision-Language Models [2.5754366051855837]
  We develop a causal pipeline to localize and test sparse visual thought circuits in Qwen3-VL-8B.<n>We train SAEs at this layer, construct task-selective sets via an explicit rule, and perform inference time scaling and ablation while quantifying accuracy and drift.
  arXiv  Detail & Related papers  (2026-03-26T06:24:36Z)
• Partial Feedback Online Learning [88.27143767009376]
  We study a new learning protocol, termed partial-feedback online learning.<n>Each instance admits a set of acceptable labels, but the learner observes only one acceptable label per round.
  arXiv  Detail & Related papers  (2026-01-29T09:39:11Z)
• Encyclo-K: Evaluating LLMs with Dynamically Composed Knowledge Statements [78.87065404966002]
  Existing benchmarks predominantly curate questions at the question level.<n>We propose Encyclo-K, a statement-based benchmark that rethinks benchmark construction from the ground up.
  arXiv  Detail & Related papers  (2025-12-31T13:55:54Z)
• Quantifying Laziness, Decoding Suboptimality, and Context Degradation in Large Language Models [0.4511923587827302]
  Large Language Models (LLMs) often exhibit behavioral artifacts such as laziness, decoded suboptimality, and context degradation.<n>Our results indicate widespread laziness in satisfying complex multi-part instructions.<n>We suggest strategies to reduce laziness and bolster multi-instruction compliance.
  arXiv  Detail & Related papers  (2025-12-19T03:01:59Z)
• Instruction Boundary: Quantifying Biases in LLM Reasoning under Various Coverage [34.247904738521136]
  We investigate how different instruction formats can either facilitate or mislead LLM reasoning ability.<n>We introduce the concept of Instruction Boundary, which systematically analyzes how different levels of prompt coverage can lead to reasoning biases.<n>We propose BiasDetector, a unified framework that quantifies LLMs' ability to identify sparse labels under different kinds of Instruction Boundary conditions.
  arXiv  Detail & Related papers  (2025-09-24T16:15:26Z)
• SHINE: Saliency-aware HIerarchical NEgative Ranking for Compositional Temporal Grounding [52.98133831401225]
  Temporal grounding, also known as video moment retrieval, aims at locating video segments corresponding to a given query sentence. We propose a large language model-driven method for negative query construction, utilizing GPT-3.5-Turbo. We introduce a coarse-to-fine saliency ranking strategy, which encourages the model to learn the multi-granularity semantic relationships between videos and hierarchical negative queries.
  arXiv  Detail & Related papers  (2024-07-06T16:08:17Z)
• Making Large Language Models Better Reasoners with Alignment [57.82176656663245]
  Reasoning is a cognitive process of using evidence to reach a sound conclusion. Recent studies reveal that fine-tuning LLMs on data with the chain of thought (COT) reasoning process can significantly enhance their reasoning capabilities. We introduce an textitAlignment Fine-Tuning (AFT) paradigm, which involves three steps.
  arXiv  Detail & Related papers  (2023-09-05T11:32:48Z)
• Hybrid Relation Guided Set Matching for Few-shot Action Recognition [51.3308583226322]
  We propose a novel Hybrid Relation guided Set Matching (HyRSM) approach that incorporates two key components. The purpose of the hybrid relation module is to learn task-specific embeddings by fully exploiting associated relations within and cross videos in an episode. We evaluate HyRSM on six challenging benchmarks, and the experimental results show its superiority over the state-of-the-art methods by a convincing margin.
  arXiv  Detail & Related papers  (2022-04-28T11:43:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.