Sealing the Audit-Runtime Gap for LLM Skills

• URL: http://arxiv.org/abs/2605.05274v1
• Date: Wed, 06 May 2026 14:23:22 GMT
• Title: Sealing the Audit-Runtime Gap for LLM Skills
• Abstract summary: We present SIGIL, the first framework that seals the audit-runtime gap for LLM skills.<n>We evaluate SIGIL on a real-world deployment against 1,023 in-the-wild skills spanning six attack types.
• Score: 22.86837061055991
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large language model (LLM) ecosystems such as Claude Code and ChatGPT increasingly rely on skills: packages of natural-language instructions and executable tools. Once in the LLM's context, skill content cannot be reliably separated from trusted instructions, and a skill's executable side can invoke privileged actions, exposing the skill supply chain to injection, tampering, and rug-pull attacks. Existing defenses are stage-bound: centralized signing, audit reports unbound from the runtime artifact, or policy engines that cannot attest to what was approved. We present SIGIL, the first framework that seals the audit-runtime gap for LLM skills. SIGIL delivers verifiable hosting through a tamper-evident, decentralized on-chain registry from which LLMs fetch skills directly. The registry admits four publication types, Transparent, Licensed, Sealed, and Committed, spanning plaintext public distribution, monetized access, custodial use, and off-chain workflows; before admission, every skill is vetted by a Decentralized Autonomous Organization (DAO) audit committee that supports pluggable auditing methods under a stake-and-slash economic model. At load time, SIGIL delivers verified loading through a skill verification protocol executed by a Skill Verification Loader (SVL) embedded as the mandatory loading path: the SVL retrieves and decrypts the skill as its type requires, verifies its integrity against the on-chain record, and enforces its permission manifest before context injection. We evaluate SIGIL on a real-world deployment against 1,023 in-the-wild skills spanning six attack types. At load time, the SVL verifies each skill's integrity against its on-chain record and enforces its approved permission manifest, completing batched verification under 86 ms. Together, these results show that LLM skills can be cryptographically bound from publication through runtime at practical cost.

Related papers

• SkillGate: Cost Efficient Runtime Malicious Skill File Detection in Coding Agents [9.553517006800913]
  Software engineering teams now deploy AI coding agents as first-class productivity tools.<n>Malicious skill files can silently reprogram agent behavior, exfiltrating credentials, injecting backdoors into generated code, or redirecting agent actions to attacker-controlled endpoints.<n>We present SkillGate, a deployable security gateway that screens AI skill packages before coding agent installation.
  arXiv  Detail & Related papers  (2026-07-28T11:50:19Z)
• OpenSkill: Open-World Self-Evolution for LLM Agents [57.88680042818572]
  We study open-world self-evolution, where an agent must build both its skills and its own verification signals from scratch.<n>We propose OpenSkill, a framework that bootstraps this loop.<n>OpenSkill attains the best automated pass rate while satisfying the no-supervision constraint.
  arXiv  Detail & Related papers  (2026-06-04T21:55:48Z)
• VASO: Formally Verifiable Self-Evolving Skills for Physical AI Agents [57.240036084348354]
  We introduce VASO, a framework for verification-guided self-evolution of robot skill contracts.<n>VASO verifies logically inconsistent skill contracts, then verifies plans induced by the skill against global and local temporal specifications.<n>On Clearpath Jackal and PX4 quadcopter tasks, VASO reaches 97.2% formal-specification compliance using fewer than 100 optimization samples.
  arXiv  Detail & Related papers  (2026-06-03T20:02:35Z)
• Harnessing LLM Agents with Skill Programs [58.356514745548026]
  We introduce HASP, a new framework that upgrades skills into executable Program Functions (PFs)<n>PFs act as executable guardrails that activate on failure-prone states and modify the next action or inject corrective context.<n> HASP drives substantial gains compared to both training-free and training-based methods on web-search, math reasoning, and coding tasks.
  arXiv  Detail & Related papers  (2026-05-18T01:35:11Z)
• Securing LLM Agents Need Intent-to-Execution Integrity [49.490963596514185]
  We argue that securing LLM agents requires defining an end-to-end correctness property that specifies when an agent's execution faithfully reflects the user's intent.<n>We observe that LLM agents are structurally analogous to compilers, where security violations correspond to mis-executions that do not preserve user intent.<n>We derive four integrity properties that must hold simultaneously: emphTool Integrity, emphInstruction Integrity, emphJudgment Integrity, and emphData Flow Integrity.
  arXiv  Detail & Related papers  (2026-05-16T12:53:31Z)
• Semia: Auditing Agent Skills via Constraint-Guided Representation Synthesis [11.214847720120192]
  We present Semia, a static auditor for agent skills.<n>Semia lifts each skill into the Skill Description Language (SDL), a Datalog fact base.<n>We evaluate Semia on 13,728 real-world skills from public marketplaces.
  arXiv  Detail & Related papers  (2026-05-01T00:48:47Z)
• Chain-of-Authorization: Internalizing Authorization into Large Language Models via Reasoning Trajectories [21.063230721511726]
  Chain-of-Authorization (CoA) framework internalizes authorization logic into Large Language Models' core capabilities.<n>CoA integrates policy execution with task responses, making authorization a causal prerequisite for substantive responses.
  arXiv  Detail & Related papers  (2026-03-24T07:13:01Z)
• Towards Verifiably Safe Tool Use for LLM Agents [53.55621104327779]
  Large language model (LLM)-based AI agents extend capabilities by enabling access to tools such as data sources, APIs, search engines, code sandboxes, and even other agents.<n>LLMs may invoke unintended tool interactions and introduce risks, such as leaking sensitive data or overwriting critical records.<n>Current approaches to mitigate these risks, such as model-based safeguards, enhance agents' reliability but cannot guarantee system safety.
  arXiv  Detail & Related papers  (2026-01-12T21:31:38Z)
• Privacy-Preserving AI-Enabled Decentralized Learning and Employment Records System [2.756161833954979]
  Learning and Employment Record (LER) systems are emerging as critical infrastructure for securely compiling and sharing educational and work achievements.<n>Existing blockchain-based platforms leverage verifiable credentials but typically lack automated skill-credential generation and the ability to incorporate unstructured evidence of learning.<n>This paper proposes a privacy-preserving, AI-enabled decentralized LER system to address these gaps.
  arXiv  Detail & Related papers  (2026-01-06T05:18:03Z)
• Agentic AI for Autonomous Defense in Software Supply Chain Security: Beyond Provenance to Vulnerability Mitigation [0.0]
  The current paper includes an example of agentic artificial intelligence (AI) based on autonomous software supply chain security.<n>It combines large language model (LLM)-based reasoning, reinforcement learning (RL), and multi-agent coordination.<n>Results show that agentic AI can facilitate the transition to self defending, proactive software supply chains.
  arXiv  Detail & Related papers  (2025-12-29T14:06:09Z)
• LLMs as verification oracles for Solidity [1.3887048755037537]
  This paper provides the first systematic evaluation of GPT-5, a state-of-the-art reasoning LLM, in this role.<n>We benchmark its performance on a large dataset of verification tasks, compare its outputs against those of established formal verification tools, and assess its practical effectiveness in real-world auditing scenarios.<n>Our study suggests a new frontier in the convergence of AI and formal methods for secure smart contract development and auditing.
  arXiv  Detail & Related papers  (2025-09-23T15:32:13Z)
• Beyond Next Token Probabilities: Learnable, Fast Detection of Hallucinations and Data Contamination on LLM Output Distributions [60.43398881149664]
  We introduce LOS-Net, a lightweight attention-based architecture trained on an efficient encoding of the LLM Output Signature.<n>It achieves superior performance across diverse benchmarks and LLMs, while maintaining extremely low detection latency.
  arXiv  Detail & Related papers  (2025-03-18T09:04:37Z)
• Get my drift? Catching LLM Task Drift with Activation Deltas [55.75645403965326]
  Task drift allows attackers to exfiltrate data or influence the LLM's output for other users.<n>We show that a simple linear classifier can detect drift with near-perfect ROC AUC on an out-of-distribution test set.<n>We observe that this approach generalizes surprisingly well to unseen task domains, such as prompt injections, jailbreaks, and malicious instructions.
  arXiv  Detail & Related papers  (2024-06-02T16:53:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.