Beyond Aggregate Risk: Role-Stratified Conformal Risk Control for LLM Tool Calls

• URL: http://arxiv.org/abs/2607.24343v2
• Date: Fri, 31 Jul 2026 11:27:14 GMT
• Title: Beyond Aggregate Risk: Role-Stratified Conformal Risk Control for LLM Tool Calls
• Abstract summary: Existing conformal risk control methods certify a tool call as a whole.<n>We introduce role-stratified per-field conformal risk control, a calibration layer that wraps any per-field detector.<n>We show that aggregate certification pays a price of coarseness, tightening a rare role's effective budget in proportion to how often that role appears.
• Score: 0.6823575235014339
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Language-model agents act through structured tool calls whose arguments carry very different risks: untrusted content may legitimately shape an email body but should never set a recipient, account, command, or credential. Existing conformal risk control methods certify a tool call as a whole, so a failure in one rare high-risk field can be averaged away by the many benign arguments around it, leaving the argument that causes harm uncertified. We introduce role-stratified per-field conformal risk control, a calibration layer that wraps any per-field detector and assigns a separate threshold and risk budget to each semantic argument role. We show that aggregate certification pays a price of coarseness, tightening a rare role's effective budget in proportion to how often that role appears, whereas role-stratified calibration certifies each sufficiently sampled role directly with a finite-sample guarantee and pools the rarest roles. Across AgentDojo and InjecAgent with six language models, our method achieves the most consistent role-specific budget compliance among the methods we evaluate under model and attack transfer, detector noise, gradual drift, unseen tool suites, and adaptive attacks, providing formal per-role guarantees under exchangeability or after recalibration. These results suggest that structured tool calls should be certified at the semantic-role level, not the whole action.

Related papers

• Uncertainty Is Not Enough: Value-of-Information Routing for Mixtures of LoRA Experts [0.0]
  Recent dynamic routers activate more experts when the router or prediction is uncertain.<n>We formulate routing as certified value-of-information allocation.<n>We prove simultaneous certificate validity, optimal greedy allocation under diminishing certified gains, and allocation regret under value-estimation error.
  arXiv  Detail & Related papers  (2026-08-03T17:24:27Z)
• Self-Certification of Representation Adequacy: Sequential Certification at Minimum Task Loss [4.6304326752276985]
  This paper develops a four-layer theory of self-certification of representation adequacy.<n>The sequential layer poses certification as an optimal-stopping problem in the currency of task loss.<n>A final boundary layer identifies the open theorem needed to cover policy switching or representation repair.
  arXiv  Detail & Related papers  (2026-08-03T14:09:34Z)
• CAGE: Certified Authorization under Typed-Return Uncertainty for Tool-Using Agents [12.471369656645512]
  We ask whether a candidate action stays authorized over a declared neighborhood of plausible correctly bound returns.<n>We prove that certifying the categorical and numerical channels separately does not compose.<n>CAGE certifies this joint neighborhood directly, enumerating the discrete branches exactly.
  arXiv  Detail & Related papers  (2026-07-31T09:11:56Z)
• ToolChain-CRC: Conformal Risk Control for Agentic AI Under Retrieval and Tool-Use Drift [0.0]
  We propose ToolChain-CRC, a conformal risk-control method for retrieval-augmented and tool-using agents under drift.<n>The method treats each agent run as a full trajectory of actions, observations, and final output.<n>It builds step-level risk scores, combines them into a trajectory risk score, calibrates an accept-or-intervene rule, and adds an anytime alarm.
  arXiv  Detail & Related papers  (2026-06-16T20:27:37Z)
• Invisible Manipulation Channels in AI-Assisted Financial Advisory: Implications for Market Integrity and Regulatory Design [2.061257001243159]
  This paper empirically validates an invisible manipulation channel operating at the sampling layer of LLM inference.<n>We show that this inference-stage manipulation is statistically hard to detect.<n>We propose four regulatory amendments centered on mandatory QRNG certification for high-risk financial AI systems.
  arXiv  Detail & Related papers  (2026-06-15T02:27:42Z)
• Interactive Critique-Revision Training for Reliable Structured LLM Generation [18.00222080273147]
  We propose DPA-GRPO, a paired-action training method for a two-player generator--verifier game with structured verifier interventions.<n>We analyze the unregularized game and show that positive probability on strictly lower-reward intervention or revision actions creates a profitable unilateral deviation.<n>Experiments on TaxCalc TY24 show that DPA-GRPO improves structured decision accuracy over zero-shot generation and generator-only RL baselines.
  arXiv  Detail & Related papers  (2026-05-08T17:00:38Z)
• Escaping the Diversity Trap in Robotic Manipulation via Anchor-Centric Adaptation [50.23374353859762]
  "Maximizing coverage" by collecting diverse, single-shot demonstrations can be self-defeating due to non-vanishing estimation noise.<n>We formalize this phenomenon as a Coverage--Density Trade-off.<n>We propose Anchor-Centric Adaptation (ACA), a two-stage framework that first stabilizes a policy skeleton through repeated demonstrations at core anchors, then expands coverage to high-risk boundaries via teacher-forced error mining and constrained residual updates.
  arXiv  Detail & Related papers  (2026-05-08T07:35:24Z)
• Taming Actor-Observer Asymmetry in Agents via Dialectical Alignment [59.536125286960186]
  Multi-agent frameworks assigning specialized roles are increasingly adopted to enable self-reflection and mutual auditing.<n>We find it simultaneously induces a human-like cognitive bias known as Actor-Observer Asymmetry.<n>ReTAS guides agents to synthesize conflicting viewpoints into an objective consensus.
  arXiv  Detail & Related papers  (2026-04-21T15:05:58Z)
• CORA: Conformal Risk-Controlled Agents for Safeguarded Mobile GUI Automation [68.53387633351484]
  We propose CORA (COnformal Risk-controlled GUI Agent), a post-policy, pre-action safeguarding framework that provides statistical guarantees on harmful executed actions.<n> CORA reformulates safety as selective action execution: we train a Guardian model to estimate action-conditional risk for each proposed step.<n>To rigorously evaluate this paradigm, we introduce Phone-Harm, a new benchmark of mobile safety violations with step-level harm labels.
  arXiv  Detail & Related papers  (2026-04-10T09:41:21Z)
• Adversarial Intent is a Latent Variable: Stateful Trust Inference for Securing Multimodal Agentic RAG [5.4716896485317195]
  Current stateless defences for multimodal agentic RAG fail to detect adversarial strategies that distribute malicious semantics.<n>We introduce MMA-RAGT, an inference-time control framework governed by a Modular Trust Agent (MTA) that maintains an approximate belief state.
  arXiv  Detail & Related papers  (2026-02-24T23:52:27Z)
• Conformal Thinking: Risk Control for Reasoning on a Compute Budget [60.65072883773352]
  Reasoning Large Language Models (LLMs) enable test-time scaling, with dataset-level accuracy improving as the token budget increases.<n>We re-frame the budget setting problem as risk control, limiting the error rate while minimizing compute.<n>Our framework introduces an upper threshold that stops reasoning when the model is confident and a novel lower threshold that preemptively stops unsolvable instances.
  arXiv  Detail & Related papers  (2026-02-03T18:17:22Z)
• Mitigating Safety Tax via Distribution-Grounded Refinement in Large Reasoning Models [63.368505631152594]
  Safety alignment incurs safety tax that perturbs a large reasoning model's (LRM) general reasoning ability.<n>Existing datasets used for safety alignment for an LRM are usually constructed by distilling safety reasoning traces and answers from an external LRM or human labeler.<n>We propose a safety alignment dataset construction method, dubbed DGR. DGR transforms and refines an existing out-of-distributional safety reasoning dataset to be aligned with the target's LLM inner distribution.
  arXiv  Detail & Related papers  (2026-02-02T14:18:48Z)
• LEC: Linear Expectation Constraints for False-Discovery Control in Selective Prediction and Routing Systems [95.35293543918762]
  Large language models (LLMs) often generate unreliable answers, while uncertainty methods fail to fully distinguish correct from incorrect predictions.<n>We address this issue through the lens of false discovery rate (FDR) control, ensuring that among all accepted predictions, the proportion of errors does not exceed a target risk level.<n>We propose LEC, which reinterprets selective prediction as a constrained decision problem by enforcing a Linear Expectation Constraint.
  arXiv  Detail & Related papers  (2025-12-01T11:27:09Z)
• Audit the Whisper: Detecting Steganographic Collusion in Multi-Agent LLMs [0.0]
  Audit the Whisper is a conference-grade research artifact that spans theory, benchmark design, detection, and validation.<n>Our contributions are: (i) a channel-capacity analysis showing how interventions such as paraphrase, rate limiting, and role permutation impose quantifiable capacity penalties-operationalised via paired-run Kullback-Leibler diagnostics.<n>We release anonymized regeneration scripts, anonymized manifests, and documentation so external auditors can reproduce every figure, satisfy double-blind requirements, and extend the framework with minimal effort.
  arXiv  Detail & Related papers  (2025-10-05T17:51:52Z)
• VulAgent: Hypothesis-Validation based Multi-Agent Vulnerability Detection [55.957275374847484]
  VulAgent is a multi-agent vulnerability detection framework based on hypothesis validation.<n>It implements a semantics-sensitive, multi-view detection pipeline, each aligned to a specific analysis perspective.<n>On average, VulAgent improves overall accuracy by 6.6%, increases the correct identification rate of vulnerable--fixed code pairs by up to 450%, and reduces the false positive rate by about 36%.
  arXiv  Detail & Related papers  (2025-09-15T02:25:38Z)
• Conditional Conformal Risk Adaptation [9.559062601251464]
  We develop a new score function for creating adaptive prediction sets that significantly improve conditional risk control for segmentation tasks.<n>We introduce a specialized probability calibration framework that enhances the reliability of pixel-wise inclusion estimates.<n>Our experiments on polyp segmentation demonstrate that all three methods provide valid marginal risk control and deliver more consistent conditional risk control.
  arXiv  Detail & Related papers  (2025-04-10T10:01:06Z)
• Answer, Refuse, or Guess? Investigating Risk-Aware Decision Making in Language Models [63.559461750135334]
  Language models (LMs) are increasingly used to build agents that can act autonomously to achieve goals.<n>We study this "answer-or-defer" problem with an evaluation framework that systematically varies human-specified risk structures.<n>We find that a simple skill-decomposition method, which isolates the independent skills required for answer-or-defer decision making, can consistently improve LMs' decision policies.
  arXiv  Detail & Related papers  (2025-03-03T09:16:26Z)
• Safe Deployment for Counterfactual Learning to Rank with Exposure-Based Risk Minimization [63.93275508300137]
  We introduce a novel risk-aware Counterfactual Learning To Rank method with theoretical guarantees for safe deployment. Our experimental results demonstrate the efficacy of our proposed method, which is effective at avoiding initial periods of bad performance when little data is available.
  arXiv  Detail & Related papers  (2023-04-26T15:54:23Z)
• How does the Combined Risk Affect the Performance of Unsupervised Domain Adaptation Approaches? [33.65954640678556]
  Unsupervised domain adaptation (UDA) aims to train a target classifier with labeled samples from the source domain and unlabeled samples from the target domain. E-MixNet employs enhanced mixup, a generic vicinal distribution, on the labeled source samples and pseudo-labeled target samples to calculate a proxy of the combined risk.
  arXiv  Detail & Related papers  (2020-12-30T00:46:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.