PCTD: Preference-Guided Counterfactual Task Decomposition for Agent Tool Retrieval

• URL: http://arxiv.org/abs/2607.15696v1
• Date: Fri, 17 Jul 2026 07:15:41 GMT
• Title: PCTD: Preference-Guided Counterfactual Task Decomposition for Agent Tool Retrieval
• Abstract summary: Task decomposition aims to transform ambiguous instructions into executable atomic subtasks.<n>Rewards for task decomposition can easily induce reward hacking in reinforcement learning-based methods.<n>We propose PCTD, a Preference-guided Counterfactual Task Decomposition framework.
• Score: 32.19528642807244
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Task decomposition aims to transform ambiguous instructions into executable atomic subtasks, thereby guiding high-precision tool retrieval. However, our analysis reveals that directly adopting tool retrieval metrics, i.e., Recall or NDCG, as rewards for task decomposition can easily induce reward hacking in reinforcement learning-based methods. Specifically, models tend to maximize retrieval matching through strategies such as repetitive decomposition. This spurious correlation between the shallow features of decomposition results and retrieval metric impairs generalization in Out-of-Domain (OOD) scenarios involving unseen tools. To address this issue, we propose PCTD, a Preference-guided Counterfactual Task Decomposition framework. PCTD quantifies the marginal causal gain of decomposition on retrieval ranking through a counterfactual reward, thereby cutting off spurious correlations at their source. Meanwhile, it introduces a preference reward to impose fine-grained structural supervision on logical coherence and atomicity, encouraging the model to generate high-quality decompositions. In addition, we construct MTDTool, the task decomposition benchmark specifically designed for mobile multi-turn interactions. Extensive experiments demonstrate that PCTD alleviates repetitive decomposition and surpasses SOTA methods in retrieval, decomposition quality, and OOD generalization.

Related papers

• Solver-Aware Decompositions for Programming-by-Example: When Dividing Requires Knowing how to Conquer [8.770341884978981]
  Decomposition-based Programming-by-example (PBE) scales performance by splitting tasks into subtasks that a learned synthesizer solves.<n>Current approaches train the decomposer to imitate ground-truth ( GT) subgoals, implicitly treating decomposition quality as intrinsic to the task.<n>We propose Solver-Aware Decomposition (SAD), a training framework that retains supervised training on GT subgoals as a structural scaffold.
  arXiv  Detail & Related papers  (2026-08-04T10:57:31Z)
• MagicSelector: Joint Optimization for Agent Tool Selection via Counterfactual Decomposition and Progressive Reranking [31.430385887150972]
  MagicSelector is a framework capable of translating ambiguous user instructions into executable atomic subtasks and guiding high-precision tool retrieval.<n>We empower MagicSelector with capabilities through three key contributions: (1) a preference-guided counterfactual task decomposition mechanism; (2) a progressive tool reranking method driven by self-distillation hard negative mining; and (3) a dual semantic boundary-aware dynamic Top-K strategy.<n>Experiments demonstrate that MagicSelector significantly outperforms state-of-the-art methods in terms of tool retrieval accuracy, OOD generalization capability, and overall token efficiency.
  arXiv  Detail & Related papers  (2026-07-20T09:44:01Z)
• OrthoTryOn: Geometric Orthogonalization for Conflict-Free Unified Fashion Generation [43.40052085990979]
  Unified fashion generation integrates tasks like virtual try-on and garment reconstruction into a single model to reduce task-specific adaptation costs.<n>We propose OrthoTryOn, a unified framework mitigating this interference within a shared Low- AdaptationRank (LoRA) module.
  arXiv  Detail & Related papers  (2026-06-26T09:23:25Z)
• Toward Trustworthy AI: Multi-Target Adversarial Attacks and Robust Defenses for Continuous Data Summarization [8.957420305238177]
  adversarial perturbations to the summarization process can compromise trustworthy AI in an upstream manner.<n>In this paper, we study adversarial attacks on continuous data summarization under similarity-level perturbations through DR-submodular optimization.<n>We show that a class of multi-resolution image summarization objectives can be formulated as multilinear extensions of non-negative submodular set functions.<n>We then formulate multi-target attack generation as a min-max problem, where one admissible perturbation of the similarity structure is optimized to degrade multiple target summarization models.
  arXiv  Detail & Related papers  (2026-06-10T08:38:27Z)
• TIER: Trajectory-Invariant Execution Rewards for Multi-Step Tool Composition [62.56752617853322]
  Outcome-based rewards provide only sparse feedback, while trajectory-supervised rewards depend on annotated reference solutions.<n>We propose TIER: Trajectory-Invariant Execution Rewards, a reward framework that derives supervision directly from function schemas and runtime execution.
  arXiv  Detail & Related papers  (2026-05-16T03:47:26Z)
• RubricEM: Meta-RL with Rubric-guided Policy Decomposition beyond Verifiable Rewards [76.17893114021757]
  Deep research systems that plan, search, evaluate evidence, and synthesize long-form reports lack ground-truth answers and span many tool-augmented decisions.<n>In this work, we argue that rubrics should serve not merely as final-answer evaluators, but as the shared interface that structures policy execution, judge feedback, and agent memory.<n>We introduceEM, a rubric-guided reinforcement learning framework that combines stagewise policy decomposition with reflection-based meta-policy evolution.
  arXiv  Detail & Related papers  (2026-05-11T17:40:38Z)
• TopoCurate:Modeling Interaction Topology for Tool-Use Agent Training [53.93696896939915]
  Training tool-use agents typically rely on Supervised Fine-Tuning (SFT) on successful trajectories and Reinforcement Learning (RL) on pass-rate-selected tasks.<n>We propose TopoCurate, an interaction-aware framework that projects multi-trial rollouts from the same task into a unified semantic quotient topology.<n>TopoCurate achieves consistent gains of 4.2% (SFT) and 6.9% (RL) over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2026-03-02T10:38:54Z)
• Beyond Single-Shot: Multi-step Tool Retrieval via Query Planning [6.212994999785976]
  TOOLQP is a lightweight framework that models retrieval as iterative query planning.<n>It decomposes instructions into sub-tasks and dynamically generates queries to interact with the retriever.<n>It achieves state-of-the-art performance, exhibiting superior zero-shot generalization, robustness across diverse retrievers, and significant improvements in downstream agentic execution.
  arXiv  Detail & Related papers  (2026-01-12T17:58:39Z)
• Beyond Monolithic Architectures: A Multi-Agent Search and Knowledge Optimization Framework for Agentic Search [56.78490647843876]
  Agentic search has emerged as a promising paradigm for complex information seeking by enabling Large Language Models (LLMs) to interleave reasoning with tool use.<n>We propose bfM-ASK, a framework that explicitly decouples agentic search into two complementary roles: Search Behavior Agents, which plan and execute search actions, and Knowledge Management Agents, which aggregate, filter, and maintain a compact internal context.
  arXiv  Detail & Related papers  (2026-01-08T08:13:27Z)
• Decomposition-Enhanced Training for Post-Hoc Attributions In Language Models [64.49342399229529]
  We argue that post-hoc attribution can be reframed as a reasoning problem, where answers are decomposed into constituent units, each tied to specific context.<n>We introduce DecompTune, a post-training method that teaches models to produce answer decompositions as intermediate reasoning steps.<n>Across extensive experiments and ablations, DecompTune substantially improves attribution quality, outperforming prior methods and matching or exceeding state-of-the-art frontier models.
  arXiv  Detail & Related papers  (2025-10-29T17:58:59Z)
• RGMDT: Return-Gap-Minimizing Decision Tree Extraction in Non-Euclidean Metric Space [28.273737052758907]
  We introduce an upper bound on the return gap between the oracle expert policy and an optimal decision tree policy. This enables us to recast the DT extraction problem into a novel non-euclidean clustering problem over the local observation and action values space of each agent. We also propose the Return-Gap-Minimization Decision Tree (RGMDT) algorithm, which is a surprisingly simple design and is integrated with reinforcement learning.
  arXiv  Detail & Related papers  (2024-10-21T21:19:49Z)
• Truncating Trajectories in Monte Carlo Policy Evaluation: an Adaptive Approach [51.76826149868971]
  Policy evaluation via Monte Carlo simulation is at the core of many MC Reinforcement Learning (RL) algorithms. We propose as a quality index a surrogate of the mean squared error of a return estimator that uses trajectories of different lengths. We present an adaptive algorithm called Robust and Iterative Data collection strategy Optimization (RIDO)
  arXiv  Detail & Related papers  (2024-10-17T11:47:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.