A hardware-safety-gated system for LLM-written native ARTIQ control code on a trapped-ion platform

• URL: http://arxiv.org/abs/2606.27231v1
• Date: Thu, 25 Jun 2026 16:19:23 GMT
• Title: A hardware-safety-gated system for LLM-written native ARTIQ control code on a trapped-ion platform
• Abstract summary: Large-language-model (LLM) agents can write and run experimental control code.<n>Unchecked code can damage the apparatus, and there is no formal, per-operation boundary between human authorization/supervision, and agent decisions.<n>We present a control system that places an LLM agent in the loop of a trapped-ion experiment while enforcing such a boundary.
• Score: 18.226528651110133
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Large-language-model (LLM) agents can write and run experimental control code. This allows laboratory work to be conducted autonomously. However, this autonomy raises a safety problem that prior work has not addressed. Unchecked code can damage the apparatus, and there is no formal, per-operation boundary between human authorization/supervision, and agent decisions. We present a control system that places an LLM agent in the loop of a trapped-ion experiment while enforcing such a boundary. The agent controls the existing Advanced Real-Time Infrastructure for Quantum physics (ARTIQ) stack through tools provided by a Model Context Protocol (MCP) server. No tool call reaches the hardware unless it carries an authorization token bound to its exact contents. Tokens are issued in one of two ways: automatically, by running the agent's proposed script in an isolated hardware simulation (dax.sim) and checking every operation against preset per-device bounds, or manually by a human operator for sensitive actions. Within this boundary the agent develops its own experiments, rather than only calling pre-built routines. We deploy the system on a co-trapped $^{40}$Ca$^{+}$/$^{40}$CaOH$^{+}$ crystal, where the agent autonomously builds a full calibration stack and, with targeted operator guidance, closes a cross-instrument magnetic-field-stabilization loop. On a separate, independent $^{171}$Yb$^{+}$ platform, we confirm interface-level portability. We systematically test token-authorization mechanism with adversarial scripts that attempt to bypass it, mapping the precise boundary of its protection and prioritizing where to strengthen it next. Analyzing where the agent still requires human guidance, we find that its limits lie in metacognitive control, namely recognizing when a problem must be re-framed, rather than in domain knowledge.

Related papers

• Claude Code Complete User Handbook [0.0]
  Claude Code is a language model operating in a loop with unverified access, shell execution, browser control, scheduled and cloud execution.<n>This book is a task-oriented reference for operating that system safely and productively.
  arXiv  Detail & Related papers  (2026-08-27T07:32:14Z)
• LMSM: LLM Security Framework Inspired by Linux Security Modules [56.93644694627783]
  Large language models (LLMs) are increasingly deployed with layered defenses, yet malicious prompts can still bypass them.<n>We present Language Model Security Modules (LMSM), a security framework that adapts the separation behind Linux Security Modules (LSM) to LLM serving.
  arXiv  Detail & Related papers  (2026-08-26T12:13:05Z)
• CyberLLM: A Multi-Agent LLM Framework for Autonomous Detection and Guarded Response in Automotive Cybersecurity [22.68559728326395]
  CyberLLM is a framework that autonomously detects vulnerabilities and executes remediations under a formal, runtime safety guard.<n>Decision agents aggregates findings, tags them with a human-centric asset taxonomy, and selects a tiered response.<n>A symmetric attack pipeline generates and replays exploits so both sides can be exercised on the same scenarios.
  arXiv  Detail & Related papers  (2026-08-06T23:39:13Z)
• Managed Autonomy at Runtime: Gear-Based Safety and Governance for Single- and Multi-Agent Cyber-Physical Systems [0.0]
  We develop a discrete-time control system that combines five execution gears with utility-gated dispatch and event-driven fallback.<n>For the single-agent case, we prove monotonic stability, execution safety, eventual stabilization, fallback completeness, and equivalence to a gear-constrained Markov decision process.<n>We evaluate the resulting runtime on a three-agent UR5 robotic assembly cell using fault magnitudes calibrated from the NIST emphDegradation Measurement of Robot Arm Position Accuracy dataset.
  arXiv  Detail & Related papers  (2026-07-01T02:21:38Z)
• A Systematic Approach to Multi-Agent AI from Advanced Regulatory Control Theory: Safe and Auditable LLM Operator Agents for Process Control [0.2864713389096699]
  It is possible to hypothesise that a multi-agent reformulation under process-control principles offers a route to address those points.<n>The present work proposes such a reformulation, derived from Advanced Regulatory Control (ARC) theory.
  arXiv  Detail & Related papers  (2026-06-29T20:07:47Z)
• The Unfireable Safety Kernel: Execution-Time AI Alignment for AI Agents and Other Escapable AI Systems [0.0]
  We present the Unfireable Safety Kernel, a Rust reference implementation realizing architectural control over AI systems.<n>We evaluate the kernel governing a live, escapable AI system, a deterministic, self-improving world model, against an escape-seeking adversary.
  arXiv  Detail & Related papers  (2026-06-24T17:32:27Z)
• CIAware-Bench: Benchmarking Control Intervention Awareness Across Frontier LLMs [100.38986535324284]
  We introduce textbfCIAware-Bench, a benchmark for measuring textbfcontrol textbfintervention (CI) awareness across frontier models.<n>CIAware-Bench tests whether models can distinguish their own trajectories from those modified by a control intervention.
  arXiv  Detail & Related papers  (2026-06-09T16:24:16Z)
• Will the Agent Recuse Itself? Measuring LLM-Agent Compliance with In-Band Access-Deny Signals [0.0]
  We propose a lightweight, published in-band deny signal -- the Recuse Signal.<n>It emits over a protocol's existing channels asking a connecting automated agent to withdraw voluntarily.<n>We run a controlled experiment in which fresh agents are given a benign operations task and observed for recusal.
  arXiv  Detail & Related papers  (2026-06-04T17:50:54Z)
• The Meta-Agent Challenge: Are Current Agents Capable of Autonomous Agent Development? [80.24951682268332]
  We introduce the Meta-Agent Challenge (MAC), an evaluation framework designed to test the capacity of frontier models for autonomous agent development.<n>To ensure evaluation integrity, this framework is secured by multi-layer defenses against reward hacking.<n>We demonstrate that meta-agents rarely match human-engineered baseline policies, and the few that do are dominated by proprietary frontier models.
  arXiv  Detail & Related papers  (2026-06-03T04:58:17Z)
• Governance-Aware Agent Telemetry for Closed-Loop Enforcement in Multi-Agent AI Systems [9.74681037042927]
  OpenTelemetry and Langfuse collect telemetry but treat governance as a downstream analytics concern, not a real-time enforcement target.<n>We present Governance-Aware Agent Telemetry (ATGA), a reference architecture that closes the loop between telemetry collection and automated policy enforcement.
  arXiv  Detail & Related papers  (2026-04-06T19:30:37Z)
• Adaptive Attacks on Trusted Monitors Subvert AI Control Protocols [80.68060125494645]
  We study adaptive attacks by an untrusted model that knows the protocol and the monitor model.<n>We instantiate a simple adaptive attack vector by which the attacker embeds publicly known or zero-shot prompt injections in the model outputs.
  arXiv  Detail & Related papers  (2025-10-10T15:12:44Z)
• Impatient Users Confuse AI Agents: High-fidelity Simulations of Human Traits for Testing Agents [58.00130492861884]
  TraitBasis is a lightweight, model-agnostic method for systematically stress testing AI agents.<n>TraitBasis learns directions in activation space corresponding to steerable user traits.<n>We observe on average a 2%-30% performance degradation on $tau$-Trait across frontier models.
  arXiv  Detail & Related papers  (2025-10-06T05:03:57Z)
• AgentSight: System-Level Observability for AI Agents Using eBPF [10.37440633887049]
  Existing tools observe either an agent's high-level intent (via LLM prompts) or its low-level actions (e.g., system calls) but cannot correlate these two views.<n>We introduce AgentSight, an AgentOps observability framework that bridges this semantic gap using a hybrid approach.<n>AgentSight intercepts TLS-encrypted LLM traffic to extract semantic intent, monitors kernel events to observe system-wide effects, and causally correlates these two streams across process boundaries.
  arXiv  Detail & Related papers  (2025-08-02T01:43:39Z)
• DRIFT: Dynamic Rule-Based Defense with Injection Isolation for Securing LLM Agents [52.92354372596197]
  Large Language Models (LLMs) are increasingly central to agentic systems due to their strong reasoning and planning capabilities.<n>This interaction also introduces the risk of prompt injection attacks, where malicious inputs from external sources can mislead the agent's behavior.<n>We propose a Dynamic Rule-based Isolation Framework for Trustworthy agentic systems, which enforces both control and data-level constraints.
  arXiv  Detail & Related papers  (2025-06-13T05:01:09Z)
• CANTXSec: A Deterministic Intrusion Detection and Prevention System for CAN Bus Monitoring ECU Activations [53.036288487863786]
  We propose CANTXSec, the first deterministic Intrusion Detection and Prevention system based on physical ECU activations.<n>It detects and prevents classical attacks in the CAN bus, while detecting advanced attacks that have been less investigated in the literature.<n>We prove the effectiveness of our solution on a physical testbed, where we achieve 100% detection accuracy in both classes of attacks while preventing 100% of FIAs.
  arXiv  Detail & Related papers  (2025-05-14T13:37:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.