Speculative Macro Commit for Faster Tool-Using Agents

• URL: http://arxiv.org/abs/2609.03236v1
• Date: Thu, 03 Sep 2026 00:31:43 GMT
• Title: Speculative Macro Commit for Faster Tool-Using Agents
• Abstract summary: We introduce textbfSpeculative Macro Commit (SMC), a runtime mechanism for a two-tier agent system.<n>SMC mines recurring multi-action skeletons from training traces and stores them in a macro library.<n>On AppWorld, SMC reduces wall time by 7.7% over SA baseline and 44.9% over sequential execution.
• Score: 18.574702996656935
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Tool-using LLM agents spend wall-clock time not only on model inference but also in serial action--observation turns, where each tool call, environment transition, and observation can delay subsequent decisions. We introduce \textbf{Speculative Macro Commit} (SMC), a runtime mechanism for a two-tier agent system: a large authoritative actor model produces the official trajectory, while a faster speculative drafter model continuously predicts and executes future action chains on an isolated environment snapshot. SMC mines recurring multi-action skeletons from training traces and stores them in a macro library used to match against action chains predicted by the drafter at runtime. When the actor's next tool call matches the first drafted action, SMC commits the remaining pre-executed draft steps, together with their observations, to the official trajectory. Using Qwen3.5-27B INT4 as the authoritative actor model and Qwen3.5-4B as the speculative drafter model, SMC matches the sequential agent's overall accuracy while reducing latency by 10.23\% over the Speculative Actions (SA) baseline and 18.59\% over sequential execution on the $τ^2$-Bench Telecom subset. On AppWorld, SMC reduces wall time by 7.7\% over SA baseline and 44.9\% over sequential execution, with a small reduction in task completion. Overall, SMC provides a practical way to reuse multi-step speculative execution and reduce agent latency beyond single-step speculative actions. Our code is publicly available \href{https://github.com/zeyuliu1037/speculative-macro-commit}{\textcolor{magenta}{here}}.

Related papers

• AOSpec: Action and Observation Co-Speculation for Low-Latency Agent Serving [12.403374013868019]
  We present AOSpec, a framework that co-speculates actions and observations across the full agent-environment loop.<n>For outcomes only execution can reveal, AOSpec launches latency-critical target actions in isolated forks.<n>We show that AOSpec reduces mean end-to-end latency by 11.8-32.5% and p99 latency by up to 42.8%.
  arXiv  Detail & Related papers  (2026-08-01T22:06:52Z)
• Agentic Coding in the Wild: Characterizing GitHub Copilot Traces at Production Scale [6.287658270510339]
  AI coding agents like GitHub Copilot, Claude Code, and Codex interleave multi-step LLM inference with tool execution.<n>We present the first production-scale characterization of this workload using sampled GitHub Copilot traces from June 2026.
  arXiv  Detail & Related papers  (2026-07-30T20:51:51Z)
• Speculate While You Reason: Teaching Agents to Predict Their Next Tool Call via Joint Agent-Speculator RL [26.192522057926737]
  Self-speculating agent solves tasks in agent mode and predicts its next tool call from partial trajectories in speculator mode.<n>Joint agent-speculator reinforcement learning method derives speculation targets from the agent's own rollouts and alternates agent and speculator updates.
  arXiv  Detail & Related papers  (2026-07-28T15:00:10Z)
• Scaling Self-Evolving Agents via Parametric Memory [69.96398842169002]
  Existing memory-augmented LLM agents store past experience exclusively in prompt space.<n>We introduce textttTMEM, a self-evolving parametric memory framework.<n>textttTMEM consistently outperforms summary-based and retrieval-based baselines across different model scales.
  arXiv  Detail & Related papers  (2026-06-03T07:18:31Z)
• IdleSpec: Exploiting Idle Time via Speculative Planning for LLM Agents [56.77101184011525]
  Large language model (LLM)-based agents solve complex tasks by leveraging multi-step reasoning with iterative tool calls and environment interactions.<n>Despite the prevalence of idle time in most agentic scenarios, existing works treat it as an unavoidable overhead.<n>We introduce IdleSpec, a scalable and generic inference approach that leverages idle-time computation to improve agent performance.
  arXiv  Detail & Related papers  (2026-05-21T08:25:17Z)
• WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation [88.10947115397971]
  This work presents WildClawBench, a native-runtime benchmark of 60 human-authored, bilingual, multimodal tasks spanning six thematic categories.<n>Each task averages roughly 8 minutes of wall-clock time and over 20 tool calls.<n> Grading is hybrid, combining deterministic rule-based checks, environment-state auditing of side effects, and an LLM/VLM judge for semantic verification.
  arXiv  Detail & Related papers  (2026-05-11T17:49:43Z)
• B-PASTE: Beam-Aware Pattern-Guided Speculative Execution for Resource-Constrained LLM Agents [0.0]
  LLM agents execute in an interleaved reasoning-and-action loop, where future tool calls cannot be launched until the current reasoning step completes.<n>We propose B-PASTE, a beam-aware extension that lifts speculation from single tools to local branch hypotheses under strict resource constraints.
  arXiv  Detail & Related papers  (2026-04-09T07:42:17Z)
• SpecEyes: Accelerating Agentic Multimodal LLMs via Speculative Perception and Planning [104.01865949020304]
  Agentic multimodal large language models (MLLMs) achieve remarkable reasoning capabilities through iterative visual tool invocation.<n>However, the cascaded perception, reasoning, and tool-calling loops introduce significant sequential overhead.<n>This overhead, termed agentic depth, incurs prohibitive latency and seriously limits system-level throughput.<n>We propose SpecEyes, an agentic-level speculative acceleration framework that breaks this sequential bottleneck.
  arXiv  Detail & Related papers  (2026-03-24T17:45:47Z)
• Act While Thinking: Accelerating LLM Agents via Pattern-Aware Speculative Tool Execution [18.50317396128381]
  PASTE is a Pattern-Aware Speculative Tool Execution method designed to hide tool latency through speculation.<n>We show that PASTE reduces average task completion time by 48.5% and improves tool execution throughput by 1.8x.
  arXiv  Detail & Related papers  (2026-03-19T13:36:50Z)
• AgentSentry: Mitigating Indirect Prompt Injection in LLM Agents via Temporal Causal Diagnostics and Context Purification [25.817251923574286]
  We propose a novel inference-time detection and mitigation framework for large language model (LLM) agents.<n>AgentSentry is the first inference-time defense to model multi-turn IPI as a temporal causal takeover.<n>We evaluate AgentSentry on the textscAgentDojo benchmark across four task suites, three IPI attack families, and multiple black-box LLMs.
  arXiv  Detail & Related papers  (2026-02-26T07:59:10Z)
• BackdoorAgent: A Unified Framework for Backdoor Attacks on LLM-based Agents [58.83028403414688]
  Large language model (LLM) agents execute tasks through multi-step workflow that combine planning, memory, and tool use.<n>Backdoor triggers injected into specific stages of an agent workflow can persist through multiple intermediate states and adversely influence downstream outputs.<n>We propose textbfBackdoorAgent, a modular and stage-aware framework that provides a unified agent-centric view of backdoor threats in LLM agents.
  arXiv  Detail & Related papers  (2026-01-08T03:49:39Z)
• SABER: Small Actions, Big Errors -- Safeguarding Mutating Steps in LLM Agents [52.20768003832476]
  We analyze execution traces on $$-Bench (Airline/Retail) and SWE-Bench Verified.<n>We formalize emphdecisive deviations, earliest action, level divergences that flip success to failure.<n>We introduce cm, a model-agnostic, gradient-free, test-time safeguard.
  arXiv  Detail & Related papers  (2025-11-26T01:28:22Z)
• Sports-Traj: A Unified Trajectory Generation Model for Multi-Agent Movement in Sports [53.637837706712794]
  We propose a Unified Trajectory Generation model, UniTraj, that processes arbitrary trajectories as masked inputs.<n>Specifically, we introduce a Ghost Spatial Masking (GSM) module, embedded within a Transformer encoder, for spatial feature extraction.<n>We benchmark three practical sports datasets, Basketball-U, Football-U, and Soccer-U, for evaluation.
  arXiv  Detail & Related papers  (2024-05-27T22:15:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.