Learning to Wait: Synchronizing Agents with the Physical World

• URL: http://arxiv.org/abs/2512.16262v1
• Date: Thu, 18 Dec 2025 07:24:44 GMT
• Title: Learning to Wait: Synchronizing Agents with the Physical World
• Abstract summary: Real-world agentic tasks often involve non-blocking actions with variable latencies, creating a fundamental textitTemporal Gap between action initiation and completion.<n>Existing environment-side solutions, such as blocking wrappers or frequent polling, either limit scalability or dilute the agent's context window with redundant observations.<n>We propose an textbfAgent-side Approach that empowers Large Language Models to actively align their textitCognitive Timeline with the physical world.
• Score: 16.592968251465475
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Real-world agentic tasks, unlike synchronous Markov Decision Processes (MDPs), often involve non-blocking actions with variable latencies, creating a fundamental \textit{Temporal Gap} between action initiation and completion. Existing environment-side solutions, such as blocking wrappers or frequent polling, either limit scalability or dilute the agent's context window with redundant observations. In this work, we propose an \textbf{Agent-side Approach} that empowers Large Language Models (LLMs) to actively align their \textit{Cognitive Timeline} with the physical world. By extending the Code-as-Action paradigm to the temporal domain, agents utilize semantic priors and In-Context Learning (ICL) to predict precise waiting durations (\texttt{time.sleep(t)}), effectively synchronizing with asynchronous environment without exhaustive checking. Experiments in a simulated Kubernetes cluster demonstrate that agents can precisely calibrate their internal clocks to minimize both query overhead and execution latency, validating that temporal awareness is a learnable capability essential for autonomous evolution in open-ended environments.

Related papers

• StreamPI: Streaming Multimodal Temporal Modeling for Vision-Language-Action Models [84.17128030384099]
  StreamPI is a streaming multimodal temporal modeling framework.<n>It equips single-frame VLA with temporal reasoning capability without introducing any additional parameters.<n> Experiments on real-robot tasks spanning memory-dependent and precise perception scenarios, as well as the simulation benchmark LIBERO, demonstrate that StreamPI outperforms pi0.5 across diverse tasks.
  arXiv  Detail & Related papers  (2026-08-26T17:33:19Z)
• ScrambleToolBench: Agents Search Exhaustively Even When Their Own Map Points to the Next Step [74.83535434786145]
  We introduce ScrambleToolBench, an interactive terminal benchmark designed to isolate behavioral reasoning.<n>The benchmark requires agents to uncover hidden tool behaviors entirely through trial-and-error interaction.<n>Our evaluation of state-of-the-art language models reveals that successful initial discovery does not translate into robust adaptation.
  arXiv  Detail & Related papers  (2026-08-03T15:07:46Z)
• Advancing Omnimodal Embodied Agents from Isolated Skills to Everyday Physical Autonomy [79.09714980186278]
  Building persistent embodied agents in unstructured environments requires unified orchestration of heterogeneous tools.<n>We present OmniAct, a framework integrating a multimodal semantic planner for skill routing across unified action spaces.<n>It achieves consistent improvements in end-to-end success across all complexity levels.
  arXiv  Detail & Related papers  (2026-06-25T16:36:35Z)
• Harmonizing Real-Time Constraints and Long-Horizon Reasoning: An Asynchronous Agentic Framework for Dynamic Scheduling [11.212756680882444]
  We introduce RACE-Sched, an agent-based framework that decouples policy execution from logical reasoning via a dual-stream architecture.<n>This approach harmonizes real-time constraints with long-horizon reasoning to achieve superior solution quality and robust adaptation to dynamic events.
  arXiv  Detail & Related papers  (2026-05-28T02:26:18Z)
• OFlow: Injecting Object-Aware Temporal Flow Matching for Robust Robotic Manipulation [55.56490813640669]
  VLA models typically act only on the current frame, while future prediction and object-aware reasoning are often learned in separate latent spaces.<n>We propose OFlow, a framework that addresses both limitations by unifying temporal foresight and object-aware reasoning in a shared semantic latent space.<n>Our method forecasts future latents with temporal flow matching, factorizes them into object-aware representations that emphasize physically relevant cues.
  arXiv  Detail & Related papers  (2026-04-20T06:38:01Z)
• AutoAgent: Evolving Cognition and Elastic Memory Orchestration for Adaptive Agents [26.380991138110925]
  AutoAgent is a self-evolving multi-agent framework built on evolving cognition, on-the-fly contextual decision-making, and elastic memory orchestration.<n>Each agent maintains structured prompt-level cognition over tools, self-capabilities, peer expertise, and task knowledge.<n>AutoAgent consistently improves task success, tool-use efficiency, and collaborative robustness over static and memory-augmented baselines.
  arXiv  Detail & Related papers  (2026-03-10T14:23:49Z)
• Agentic Spatio-Temporal Grounding via Collaborative Reasoning [80.83158605034465]
  Temporal Video Grounding aims to retrieve thetemporal tube of a target object or person in a video given a text query.<n>We propose the Agentic Spatio-Temporal Grounder (ASTG) framework for the task of STVG towards an open-world and training-free scenario.<n>Specifically, two specialized agents SRA (Spatial Reasoning Agent) and TRA (Temporal Reasoning Agent) constructed leveraging on modern Multimoal Large Language Models (MLLMs)<n>Experiments on popular benchmarks demonstrate the superiority of the proposed approach where it outperforms existing weakly-supervised and zero-shot approaches by a margin
  arXiv  Detail & Related papers  (2026-02-10T10:16:27Z)
• Real-Time Robot Execution with Masked Action Chunking [38.37108371991901]
  Real-time execution is essential for cyber-physical systems such as robots.<n>Asynchronous inference has recently emerged as a system-level paradigm for real-time robot manipulation.<n>We propose REMAC, which learns corrective adjustments on the pretrained policy through masked action chunking.
  arXiv  Detail & Related papers  (2026-01-27T23:48:32Z)
• HALO: Semantic-Aware Distributed LLM Inference in Lossy Edge Network [50.33808558714122]
  Large language models' (LLMs) inference at the edge can facilitate prompt service responsiveness while protecting user privacy.<n>We propose HALO, a novel framework that can boost the distributed LLM inference in lossy edge network.<n> Experimental results from a Raspberry Pi cluster demonstrate that HALO achieves a 3.41x end-to-end speedup for LLaMA-series LLMs under unreliable network conditions.
  arXiv  Detail & Related papers  (2026-01-16T07:37:23Z)
• Towards Efficient Agents: A Co-Design of Inference Architecture and System [66.59916327634639]
  This paper presents AgentInfer, a unified framework for end-to-end agent acceleration.<n>We decompose the problem into four synergistic components: AgentCollab, AgentSched, AgentSAM, and AgentCompress.<n>Experiments on the BrowseComp-zh and DeepDiver benchmarks demonstrate that through the synergistic collaboration of these methods, AgentInfer reduces ineffective token consumption by over 50%.
  arXiv  Detail & Related papers  (2025-12-20T12:06:13Z)
• SCOPE: Prompt Evolution for Enhancing Agent Effectiveness [53.75986399936395]
  Large Language Model (LLM) agents are increasingly deployed in environments that generate massive, dynamic contexts.<n>While agents have access to this context, their static prompts lack the mechanisms to manage it effectively.<n>We introduce textbfSCOPE (Self-evolving Context Optimization via Prompt Evolution)<n>We propose a Dual-Stream mechanism that balances tactical specificity (resolving immediate errors) with strategic generality (evolving long-term principles)
  arXiv  Detail & Related papers  (2025-12-17T12:25:05Z)
• Real-Time Inference for Distributed Multimodal Systems under Communication Delay Uncertainty [37.15356899831919]
  Connected cyber-physical systems perform inference based on real-time inputs from multiple data streams.<n>We propose a novel neuro-inspired non-blocking inference paradigm that employs adaptive temporal windows of integration.<n>Our framework achieves robust real-time inference with finer-grained control over the accuracy-latency tradeoff.
  arXiv  Detail & Related papers  (2025-11-20T10:48:54Z)
• Adaptive Reinforcement Learning for Unobservable Random Delays [46.04329493317009]
  We introduce a general framework that enables agents to adaptively handle unobservable and time-varying delays.<n>Specifically, the agent generates a matrix of possible future actions to handle both unpredictable delays and lost action packets sent over networks.<n>Our method significantly outperforms state-of-the-art approaches across a wide range of benchmark environments.
  arXiv  Detail & Related papers  (2025-06-17T11:11:37Z)
• Adaptive Deadline and Batch Layered Synchronized Federated Learning [66.93447103966439]
  Federated learning (FL) enables collaborative model training across distributed edge devices while preserving data privacy, and typically operates in a round-based synchronous manner.<n>We propose ADEL-FL, a novel framework that jointly optimize per-round deadlines and user-specific batch sizes for layer-wise aggregation.
  arXiv  Detail & Related papers  (2025-05-29T19:59:18Z)
• Asynchronous Tool Usage for Real-Time Agents [61.3041983544042]
  We introduce asynchronous AI agents capable of parallel processing and real-time tool-use. Our key contribution is an event-driven finite-state machine architecture for agent execution and prompting. This work presents both a conceptual framework and practical tools for creating AI agents capable of fluid, multitasking interactions.
  arXiv  Detail & Related papers  (2024-10-28T23:57:19Z)
• MOSS: Enabling Code-Driven Evolution and Context Management for AI Agents [7.4159044558995335]
  We introduce MOSS (llM-oriented Operating System Simulation), a novel framework that integrates code generation with a dynamic context management system. At its core, the framework employs an Inversion of Control container in conjunction with decorators to enforce the least knowledge principle. We show how this framework can enhance the efficiency and capabilities of agent development and highlight its advantages in moving towards Turing-complete agents.
  arXiv  Detail & Related papers  (2024-09-24T14:30:21Z)
• DASA: Delay-Adaptive Multi-Agent Stochastic Approximation [64.32538247395627]
  We consider a setting in which $N$ agents aim to speedup a common Approximation problem by acting in parallel and communicating with a central server. To mitigate the effect of delays and stragglers, we propose textttDASA, a Delay-Adaptive algorithm for multi-agent Approximation.
  arXiv  Detail & Related papers  (2024-03-25T22:49:56Z)
• Procedural Adherence and Interpretability Through Neuro-Symbolic Generative Agents [0.9886108751871757]
  We propose a combination of formal logic-based program synthesis and LLM content generation to bring guarantees of procedural adherence and interpretability to generative agent behavior. To illustrate the benefit of procedural adherence and interpretability, we use Temporal Stream Logic (TSL) to generate an automaton that enforces an interpretable, high-level temporal structure on an agent.
  arXiv  Detail & Related papers  (2024-02-24T21:36:26Z)
• Asynchronous Actor-Critic for Multi-Agent Reinforcement Learning [19.540926205375857]
  Synchronizing decisions across multiple agents in realistic settings is problematic since it requires agents to wait for other agents to terminate and communicate about termination reliably. We formulate a set of asynchronous multi-agent actor-critic methods that allow agents to directly optimize asynchronous policies in three standard training paradigms.
  arXiv  Detail & Related papers  (2022-09-20T16:36:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.