AgentProv: Auditing Agentic LLM API Providers via Tool-use Policy Probes

• URL: http://arxiv.org/abs/2609.00052v1
• Date: Sun, 30 Aug 2026 08:22:12 GMT
• Title: AgentProv: Auditing Agentic LLM API Providers via Tool-use Policy Probes
• Abstract summary: Commercial LLM APIs advertise a specific foundation model, but the served backbone may be silently substituted, quantized, or wrapped.<n>All existing audits decide backbone identity from the text-output channel.<n>We introduce Agentic Provenance (AgentProv), the first action-based identity audit for agentic LLM APIs.
• Score: 38.37599802008238
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Commercial LLM APIs advertise a specific foundation model, but the served backbone may be silently substituted, quantized, or wrapped, for example to save deployment costs. All existing audits decide backbone identity from the text-output channel, which is structurally fragile for agentic APIs because modern serving stacks (OpenAI, Anthropic, Gemini, Cloudflare Workers AI, LangGraph) discard text and expose only structured actions when the model calls a tool, and provider-injected system prompts can distort text distributions enough that text-channel tests falsely accuse honest providers of substituting the claimed model. We observe that recent agentic post-training internalizes tool-use directly into the weights, opening a new audit channel that the serving stack still exposes and that is largely invariant to deployment context. We introduce Agentic Provenance (AgentProv), the first action-based identity audit for agentic LLM APIs: AgentProv fingerprints a deployed model through its categorical tool-call distribution and decides identity via an MMD permutation test. AgentProv catches every substituted model (100% on 630 evaluated checkpoint pairs), while holding the false-positive rate under system-prompt injection at 7% (vs. 67% for MET and 53% for RUT). On third-party API endpoints, AgentProv's disagreements with MET are consistent with an independent token-count side-channel that detects provider-injected system prompts.

Related papers

• Zero-Knowledge Predicate Proofs Between AI Agents: A Measured, Cross-Protocol Gateway and the Source-Integrity Gap [0.0]
  Multi-agent AI platforms move quickly from staging to production, but the way agents establish trust remains rudimentary.<n>We take provable data minimisation between agents from proposal to running system.<n>A 32-bit threshold predicate proves in 6.2 ms and verifies in 1.0 ms with a 608-byte Bulletproofs proof.
  arXiv  Detail & Related papers  (2026-08-30T23:16:17Z)
• ROPE: Routed Origin Policy Enforcement against Indirect Prompt Injection [61.4148196085256]
  We present ROPE (Routed Origin Policy Enforcement), which is anchored in a structural notion of trust.<n>Our approach admits two provable guarantees: 1) at every step of a trajectory, no value whose only origin is attacker-writable content reaches an origin-guarded parameter, and 2) no rewording of an injection changes an admission decision.
  arXiv  Detail & Related papers  (2026-08-27T01:22:14Z)
• Agent-Native Telemetry: Verifiable State-Delta Evidence for Autonomous Operations [2.124730017640531]
  This paper introduces agent-native telemetry, an operational evidence architecture for autonomous machine operators.<n>We present the Agent Telemetry Protocol (ATP) and the State-Delta Evidence Ledger.
  arXiv  Detail & Related papers  (2026-08-17T06:50:12Z)
• AutoTrace: From Patches to Triggers via Agentic Interprocedural Exploration [6.39392731596656]
  Given a vulnerability-fixing commit, trigger localization asks which specific statement turns the vulnerable program state into a concrete unsafe operation.<n>We present AutoTrace, an agentic pipeline that localizes vulnerability triggers by exploring a code property graph layer by layer.<n>Agents never accept a trigger on their own authority; every reported trigger is backed by explicit evidence drawn from the graph.
  arXiv  Detail & Related papers  (2026-07-13T18:21:42Z)
• From Attack Simulation to SIEM Rule: Deterministic Detection-as-Code Synthesis with Probe-Level Traceability [51.56484100374058]
  Security teams simulate attacks against their own systems to check whether their monitoring would catch a real intruder.<n>A human bridges that gap by hand, reading each finding and writing the corresponding Sigma rule.<n>We show this translation can be partially automated when probes are drawn from a locked corpus.
  arXiv  Detail & Related papers  (2026-06-03T14:26:25Z)
• Bridging Protocol and Production: Design Patterns for Deploying AI Agents with Model Context Protocol [0.0]
  The Model Context Protocol (MCP) standardizes how AI agents discover and invoke external tools.<n>Three protocol-level primitives remain missing: identity propagation, adaptive tool budgeting, and structured error semantics.<n>This paper identifies these gaps through field lessons from an enterprise deployment of an AI agent platform integrated with a major cloud provider's MCP servers.
  arXiv  Detail & Related papers  (2026-03-12T17:42:24Z)
• DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems [48.971606069204825]
  DoVer is an intervention-driven debug framework for large language model (LLM)-based multi-agent systems.<n>It augments hypothesis generation with active verification through targeted interventions.<n>DoVer flips 18-28% of failed trials into successes, achieves up to 16% milestone progress, and validates or refutes 30-60% of failure hypotheses.
  arXiv  Detail & Related papers  (2025-12-07T09:23:48Z)
• Agentic JWT: A Secure Delegation Protocol for Autonomous AI Agents [0.6747475365990533]
  In agentic settings reasoning, prompt injection, or multi-agent orchestration can silently expand privileges.<n>We introduce Agentic JWT (A-JWT), a dual-faceted intent token that binds each agent's action to verifiable user intent.<n>A-JWT carries an agent's identity as a one-way hash derived from its prompt, tools and configuration.
  arXiv  Detail & Related papers  (2025-09-16T23:43:24Z)
• Are You Getting What You Pay For? Auditing Model Substitution in LLM APIs [71.7892165868749]
  Commercial Large Language Model (LLM) APIs create a fundamental trust problem.<n>Users pay for specific models but have no guarantee that providers deliver them faithfully.<n>We formalize this model substitution problem and evaluate detection methods under realistic adversarial conditions.<n>We propose and evaluate the use of Trusted Execution Environments (TEEs) as one practical and robust solution.
  arXiv  Detail & Related papers  (2025-04-07T03:57:41Z)
• SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraints [59.645885492637845]
  SOPBench is an evaluation pipeline that transforms each service-specific SOP code program into a directed graph of executable functions.<n>Our approach transforms each service-specific SOP code program into a directed graph of executable functions and requires agents to call these functions based on natural language SOP descriptions.<n>We evaluate 18 leading models, and results show the task is challenging even for top-tier models.
  arXiv  Detail & Related papers  (2025-03-11T17:53:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.