Agent-X: Full Pipeline Acceleration of On-device AI Agents

• URL: http://arxiv.org/abs/2605.10380v1
• Date: Mon, 11 May 2026 11:23:38 GMT
• Title: Agent-X: Full Pipeline Acceleration of On-device AI Agents
• Abstract summary: LLM-based agents deliver state-of-the-art performance across tasks but incur high end-to-end latency on edge devices.<n>We introduce Agent-X, a software-only, accuracy-preserving framework that accelerates both the prefill and decode stages of on-device agent workloads.
• Score: 3.125320161154482
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: LLM-based agents deliver state-of-the-art performance across tasks but incur high end-to-end latency on edge devices. We introduce Agent-X, a software-only, accuracy-preserving framework that accelerates both the prefill and decode stages of on-device agent workloads. Agent-X's two key components rewrite prompts to leverage prefix caching tailored to agent-specific input-token patterns and enable LLM-free speculative decoding for fast token generation with minimal overhead. On representative agentic workloads, Agent-X achieves a 1.61x end-to-end speedup in real systems with no accuracy loss and can be seamlessly integrated into existing on-device AI agents. To the best of our knowledge, ours is the first to systematically characterize and eliminate latency bottlenecks in on-device agents.

Related papers

• Multi-Agent Computer Use [72.79887808312706]
  We argue that we should move towards evaluating and building multi-agent computer use (MACU) systems.<n>We propose a general multi-agent setup in which a manager model decomposes computer use tasks as a directed acyclic graph (DAG)<n>At each iteration, the manager dispatches parallel CUA subagents to carry out nodes on the ready frontier of the DAG.
  arXiv  Detail & Related papers  (2026-06-01T01:29:36Z)
• Speculative Interaction Agents: Building Real-Time Agents with Asynchronous I/O and Speculative Tool Calling [64.40340291543971]
  We propose Speculative Interaction Agents to enable real-time interaction with small edge-scale models.<n>We also present a clock-based training methodology that adapts the model to handle streaming inputs and asynchronous responses.<n>This approach provides 1.6-2.2$times$ speedups with the Qwen2.5-3B-Instruct and Llama-3.2-3B-Instruct models across multiple tool calling benchmarks.
  arXiv  Detail & Related papers  (2026-05-13T11:20:52Z)
• OTora: A Unified Red Teaming Framework for Reasoning-Level Denial-of-Service in LLM Agents [21.836018755837085]
  OTora is a framework for instantiating Reasoning-Level Denial-of-Service attacks.<n> OTora generates agent-aware reasoning payloads via an ICL-guided genetic search.<n>It achieves up to 10 times increases in reasoning tokens and order-of-magnitude latency slowdowns.
  arXiv  Detail & Related papers  (2026-05-09T10:55:11Z)
• SkillJect: Automating Stealthy Skill-Based Prompt Injection for Coding Agents with Trace-Driven Closed-Loop Refinement [120.52289344734415]
  We propose an automated framework for stealthy prompt injection tailored to agent skills.<n>The framework forms a closed loop with three agents: an Attack Agent that synthesizes injection skills under explicit stealth constraints, a Code Agent that executes tasks using the injected skills and an Evaluate Agent that logs action traces.<n>Our method consistently achieves high attack success rates under realistic settings.
  arXiv  Detail & Related papers  (2026-02-15T16:09:48Z)
• Towards Efficient Agents: A Co-Design of Inference Architecture and System [66.59916327634639]
  This paper presents AgentInfer, a unified framework for end-to-end agent acceleration.<n>We decompose the problem into four synergistic components: AgentCollab, AgentSched, AgentSAM, and AgentCompress.<n>Experiments on the BrowseComp-zh and DeepDiver benchmarks demonstrate that through the synergistic collaboration of these methods, AgentInfer reduces ineffective token consumption by over 50%.
  arXiv  Detail & Related papers  (2025-12-20T12:06:13Z)
• Efficient Mixture-of-Agents Serving via Tree-Structured Routing, Adaptive Pruning, and Dependency-Aware Prefill-Decode Overlap [15.352230356342366]
  Mixture-of-Agents (MoA) inference can suffer from dense inter-agent communication and low hardware utilization.<n>We present a serving design that targets these bottlenecks through an algorithm-system co-design.
  arXiv  Detail & Related papers  (2025-12-19T23:06:58Z)
• Speculative Actions: A Lossless Framework for Faster Agentic Systems [6.708126506152481]
  Execution of AI agents is often slow, hampering training, evaluation, and deployment.<n>Inspired by speculative execution in microprocessors, we propose a framework that predicts likely actions using faster models.<n>We evaluate this framework across three agentic environments: gaming, e-commerce, web search, and a "lossy" extension for an operating systems environment.
  arXiv  Detail & Related papers  (2025-10-05T21:28:11Z)
• Agent.xpu: Efficient Scheduling of Agentic LLM Workloads on Heterogeneous SoC [11.82567747365518]
  This paper introduces Agent.xpu, an efficient serving system for agentic LLM workloads on memory-unified heterogeneous SOCs.<n>With dedicated offline profiling, Agent.xpu first constructs a heterogeneous execution graph, which fuses and chunks model kernels for affinity-guided, elastic accelerator mapping.<n>At runtime, its online scheduler enables fine-grained, kernel-level preemption to guarantee the responsiveness of reactive tasks.
  arXiv  Detail & Related papers  (2025-06-30T16:50:48Z)
• AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents [54.29555239363013]
  We propose a generic black-box fuzzing framework, AgentVigil, to automatically discover and exploit indirect prompt injection vulnerabilities.<n>We evaluate AgentVigil on two public benchmarks, AgentDojo and VWA-adv, where it achieves 71% and 70% success rates against agents based on o3-mini and GPT-4o.<n>We apply our attacks in real-world environments, successfully misleading agents to navigate to arbitrary URLs, including malicious sites.
  arXiv  Detail & Related papers  (2025-05-09T07:40:17Z)
• EcoAgent: An Efficient Edge-Cloud Collaborative Multi-Agent Framework for Mobile Automation [36.08217588070538]
  Cloud-based mobile agents powered by (multimodal) large language models ((M)LLMs) offer strong reasoning abilities but suffer from high latency and cost.<n>We propose textbfEcoAgent, an textbfEdge-textbfCloud ctextbfOllaborative multi-agent framework for mobile automation.<n>EcoAgent features a closed-loop collaboration among a cloud-based Planning Agent and two edge-based agents: the Execution Agent for action execution and the Observation Agent for verifying outcomes.
  arXiv  Detail & Related papers  (2025-05-08T17:31:20Z)
• Agent-as-a-Judge: Evaluate Agents with Agents [61.33974108405561]
  We introduce the Agent-as-a-Judge framework, wherein agentic systems are used to evaluate agentic systems. This is an organic extension of the LLM-as-a-Judge framework, incorporating agentic features that enable intermediate feedback for the entire task-solving process. We present DevAI, a new benchmark of 55 realistic automated AI development tasks.
  arXiv  Detail & Related papers  (2024-10-14T17:57:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.