Diagnosing Task Insensitivity in Language Agents

• URL: http://arxiv.org/abs/2606.26918v1
• Date: Thu, 25 Jun 2026 11:53:41 GMT
• Title: Diagnosing Task Insensitivity in Language Agents
• Abstract summary: Large language models can serve as capable long-horizon agents, but their out-of-distribution generalization remains weak.<n>We show that models often continue with actions aligned with the original task even when the instruction is semantically corrupted.<n>We propose Task-Perturbed NLL Optimization, a lightweight contrastive regularizer that explicitly encourages action dependence on the task instruction.
• Score: 20.018733779493587
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language models can serve as capable long-horizon agents, but their out-of-distribution (OOD) generalization remains weak. We identify a key source of this failure as task insensitivity: when faced with similar but distinct tasks, models might apply patterns learned during training and fail to solve the task at hand. We show that models often continue with actions aligned with the original task even when the instruction is semantically corrupted and cannot be directly answered. We further find that, when we replace the task description in a trained prompt with another similar but distinct task, the model may still output the same action. This behavior is accompanied by a consistent training-time attention drift away from task tokens and toward local observations, suggesting an optimization bias toward shortcuts. To mitigate this problem, we propose Task-Perturbed NLL Optimization, a lightweight contrastive regularizer that explicitly encourages action dependence on the task instruction. Extensive evaluations show that our intervention improves task sensitivity and OOD generalization while preserving more stable attention to task tokens.

Related papers

• Shared Lexical Task Representations Explain Behavioral Variability In LLMs [35.89864679533707]
  We find that despite large variation in performance as a function of the prompt, the model engages some common underlying mechanisms across different prompts of a task.<n>Specifically, we identify task-specific attention heads whose outputs literally describe the task and show that these heads are shared across prompting styles and trigger subsequent answer production.
  arXiv  Detail & Related papers  (2026-04-23T19:43:29Z)
• Task Vectors in In-Context Learning: Emergence, Formation, and Benefit [17.72043522825441]
  We investigate the formation of task vectors in a controlled setting using models trained from scratch on synthetic datasets.<n>Our findings confirm that task vectors naturally emerge under certain conditions, but the tasks may be relatively weakly and/or non-locally encoded within the model.<n>To promote strong task vectors encoded at a prescribed location within the model, we propose an auxiliary training mechanism based on a task vector prompting loss.
  arXiv  Detail & Related papers  (2025-01-16T01:54:23Z)
• Active Instruction Tuning: Improving Cross-Task Generalization by Training on Prompt Sensitive Tasks [101.40633115037983]
  Instruction tuning (IT) achieves impressive zero-shot generalization results by training large language models (LLMs) on a massive amount of diverse tasks with instructions. How to select new tasks to improve the performance and generalizability of IT models remains an open question. We propose active instruction tuning based on prompt uncertainty, a novel framework to identify informative tasks, and then actively tune the models on the selected tasks.
  arXiv  Detail & Related papers  (2023-11-01T04:40:05Z)
• Instruction Position Matters in Sequence Generation with Large Language Models [67.87516654892343]
  Large language models (LLMs) are capable of performing conditional sequence generation tasks, such as translation or summarization. We propose enhancing the instruction-following capability of LLMs by shifting the position of task instructions after the input sentences.
  arXiv  Detail & Related papers  (2023-08-23T12:36:57Z)
• Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks [71.19560970717495]
  Recent language models show impressive performance across a wide range of tasks. Are these skills general and transferable, or specialized to specific tasks seen during pretraining? We propose an evaluation framework based on "counterfactual" task variants that deviate from the default assumptions underlying standard tasks.
  arXiv  Detail & Related papers  (2023-07-05T17:50:42Z)
• Exposing and Addressing Cross-Task Inconsistency in Unified Vision-Language Models [80.23791222509644]
  Inconsistent AI models are considered brittle and untrustworthy by human users. We find that state-of-the-art vision-language models suffer from a surprisingly high degree of inconsistent behavior across tasks. We propose a rank correlation-based auxiliary training objective, computed over large automatically created cross-task contrast sets.
  arXiv  Detail & Related papers  (2023-03-28T16:57:12Z)
• ForkMerge: Mitigating Negative Transfer in Auxiliary-Task Learning [59.08197876733052]
  Auxiliary-Task Learning (ATL) aims to improve the performance of the target task by leveraging the knowledge obtained from related tasks. Sometimes, learning multiple tasks simultaneously results in lower accuracy than learning only the target task, known as negative transfer. ForkMerge is a novel approach that periodically forks the model into multiple branches, automatically searches the varying task weights.
  arXiv  Detail & Related papers  (2023-01-30T02:27:02Z)
• Task Compass: Scaling Multi-task Pre-training with Task Prefix [122.49242976184617]
  Existing studies show that multi-task learning with large-scale supervised tasks suffers from negative effects across tasks. We propose a task prefix guided multi-task pre-training framework to explore the relationships among tasks. Our model can not only serve as the strong foundation backbone for a wide range of tasks but also be feasible as a probing tool for analyzing task relationships.
  arXiv  Detail & Related papers  (2022-10-12T15:02:04Z)
• Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work? [44.88358841370665]
  It is poorly understood when and why intermediate-task training is beneficial for a given target task. We perform a large-scale study on the pretrained RoBERTa model with 110 intermediate-target task combinations. We observe that intermediate tasks requiring high-level inference and reasoning abilities tend to work best.
  arXiv  Detail & Related papers  (2020-05-01T21:49:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.