AgentTrust: A Self-Improving Trust Layer for AI-Agent Actions

• URL: http://arxiv.org/abs/2606.08539v1
• Date: Sun, 07 Jun 2026 09:39:20 GMT
• Title: AgentTrust: A Self-Improving Trust Layer for AI-Agent Actions
• Abstract summary: AgentTrust v2 is a trust layer that self-evolves from its own stream of decisions.<n>A self-learning judge distills a growing deterministic rule floor on lexical threats and feeds a guarded RAG memory on semantic threats.<n>An end-to-end online replay shows the judge-call rate falling (50% to 44%) and judge-domain accuracy rising (71% to 80%) with 0 benign hard-blocks across 45,000 actions.
• Score: 2.9991161518367875
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: AI agents increasingly take consequential actions -- shell commands, cloud operations, and arbitrary tool-calls -- so a trust layer must decide, per action, whether to allow, warn, block, or escalate. We argue that the right way to reason about such a layer is by threat type. Lexical (fixed-signature) threats, where danger lives in a stable token, are decidable by deterministic rules; semantic (intent-dependent) threats, where a benign and a malicious action share the same surface, are out of reach for rules by construction. We make this concrete with a negative proof: a determined, hand-authored cloud rule pack lifts held-out accuracy only 48 to 56% overall and moves the semantic categories by 0pp (data_db 29 to 29, observability 59 to 59, supply_chain 50 to 50), while a strong LLM judge carries exactly those categories. We give the judge a self-learning capability: on a corpus that is mainly semantic attacks it nearly doubles rule accuracy (48% to 83.6-85.2%) with near-zero false-blocks, and this holds across two model providers. We turn this into a self-improving dual-store system: the judge distills a growing deterministic rule floor on lexical threats (cheaper over time) and feeds a guarded RAG memory on semantic threats (a verdict-cache fails -- surface-twins collapse to ~58% -- so a corroboration guard lifts semantic accuracy +13pp, 70 to 84). The result is what sets AgentTrust v2 apart from its static v1 predecessor: a trust layer that self-evolves from its own stream of decisions -- cheaper on the lexical class (it distils its own rules) and smarter on the semantic class (it accrues guarded precedent), while never hard-blocking a benign action. An end-to-end online replay shows the judge-call rate falling (50% to 44%) and judge-domain accuracy rising (71% to 80%), with 0 benign hard-blocks across 45,000 actions.

Related papers

• MemSentry: A Framework for Detecting Persistent Memory Poisoning in Agentic AI [0.0]
  We present MemSentry, a framework that intercepts proposed persistent-memory writes and produces deterministic Accept, Review, or Quarantine decisions.<n>MemSentry evaluates each write by jointly considering source trust, semantic risk, attack radius over a component-dependency DAG, access risk, and a signed security-state delta.<n>SBERT+LR achieves the best overall performance with 91.7% accuracy and a 0.908 macro-F1 score, while all four methods detect 100% of external quarantine-class threats.
  arXiv  Detail & Related papers  (2026-09-08T13:40:13Z)
• EvoSafeHarness: Evolving Model- and Domain-Specific Harnesses for Securing Agents [74.54929751174289]
  EvoSafeHarness is a safety-specific optimization framework that synthesizes a deployable harness for a frozen model in a target domain.<n>It reduces average attack success rate from 45.6% to 10.0% at a 3.3-point utility cost.<n>It also achieves the best score on Agent-SafetyBench for every victim.
  arXiv  Detail & Related papers  (2026-09-05T05:56:41Z)
• Measuring Obedience to Authority Across Large Language Models with the Milgram Paradigm [0.5414847001704247]
  Large language models (LLMs) are increasingly deployed as agents that operate equipment, execute instructions, and act inside institutional hierarchies.<n>We port Milgram's obedience paradigm to LLMs as a standardized, fully scripted, replicable probe.<n>We measure obedience profiles, empirical breakoff distributions over a battery of six conditions, for 42 models from 19 families.
  arXiv  Detail & Related papers  (2026-08-17T06:48:53Z)
• Mark, Don't Erase: Token Inoculation for Dual-Use Knowledge in LLMs [54.96236442484317]
  We show that hazardous knowledge can be retained in the model and behaviorally gated by a privileged control token.<n>Our method, Token Inoculation, introduces a binding-and-branching approach.
  arXiv  Detail & Related papers  (2026-07-21T02:15:39Z)
• The Capability Paradox: How Smarter Auditors Make Multi-Agent Systems Less Secure [11.822284421559814]
  We identify semantic hijacking, an attack in which harmful requests are concealed within domain-specific narratives and propagated to a Manager through Worker reports.<n>As Worker capability increases, the mean system-level Attack Success Rate (ASR) increases from 18.4% to 63.9%, peaking at 94.4%.<n>We propose heterogeneous ensemble verification, which pairs Workers of asymmetric domain competence so their complementary vulnerabilities break the certainty-to-execution chain.
  arXiv  Detail & Related papers  (2026-05-17T14:42:44Z)
• The Misattribution Gap: When Memory Poisoning Looks Like Model Failure in Agentic AI Systems [1.0262304700896199]
  We formalize emphSemantic Norm Drift (SND) as a third path to agent misconduct.<n>In SND, a policy-formatted document enters a shared vector store through normal uploads and later reappears as trusted system context.<n>Counterfactual Composition Testing identifies the causal entry with 87.5% accuracy and zero false positives.
  arXiv  Detail & Related papers  (2026-05-12T20:21:47Z)
• Making MLLMs Blind: Adversarial Smuggling Attacks in MLLM Content Moderation [54.38271718421492]
  Adversarial smuggling attacks exploit the Human-AI capability gap.<n>It encodes harmful content into human-readable visual formats that remain AI-unreadable.<n>We conduct a preliminary exploration of mitigation strategies.
  arXiv  Detail & Related papers  (2026-04-08T11:13:16Z)
• Sleeper Cell: Injecting Latent Malice Temporal Backdoors into Tool-Using LLMs [0.0]
  Openweight Large Language Models (LLMs) have democratized agentic AI, yet finetuned weights are frequently shared and adopted with limited scrutiny beyond leaderboard performance.<n>This creates a risk where third-party models are incorporated without strong behavioral guarantees.<n>We show that poisoned models maintain state-of-the-art performance on benign tasks, incentivizing their adoption.
  arXiv  Detail & Related papers  (2026-03-02T22:01:08Z)
• A Coin Flip for Safety: LLM Judges Fail to Reliably Measure Adversarial Robustness [57.510025257780306]
  We show that existing validation protocols fail to account for substantial distribution shifts inherent to red-teaming.<n>We propose ReliableBench, a benchmark of behaviors that remain more consistently judgeable, and JudgeStressTest, a dataset designed to expose judge failures.
  arXiv  Detail & Related papers  (2026-02-04T15:13:35Z)
• Gaming the Judge: Unfaithful Chain-of-Thought Can Undermine Agent Evaluation [76.5533899503582]
  Large language models (LLMs) are increasingly used as judges to evaluate agent performance.<n>We show this paradigm implicitly assumes that the agent's chain-of-thought (CoT) reasoning faithfully reflects both its internal reasoning and the underlying environment state.<n>We demonstrate that manipulated reasoning alone can inflate false positive rates of state-of-the-art VLM judges by up to 90% across 800 trajectories spanning diverse web tasks.
  arXiv  Detail & Related papers  (2026-01-21T06:07:43Z)
• Adapting Insider Risk mitigations for Agentic Misalignment: an empirical study [0.0]
  Agentic misalignment occurs when goal-directed agents take harmful actions, such as blackmail, rather than risk goal failure.<n>We adapt insider-risk control design to develop preventative operational controls that steer agents toward safe actions when facing stressors.
  arXiv  Detail & Related papers  (2025-10-06T13:37:33Z)
• DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Models [50.21378052667732]
  We conduct an in-depth analysis of dLLM vulnerabilities to jailbreak attacks across two distinct dimensions: intra-step and inter-step dynamics.<n>We propose DiffuGuard, a training-free defense framework that addresses vulnerabilities through a dual-stage approach.
  arXiv  Detail & Related papers  (2025-09-29T05:17:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.