Retriever: Composing Closed-Loop Asynchronous Robot Programs

• URL: http://arxiv.org/abs/2607.17213v1
• Date: Sun, 19 Jul 2026 12:14:28 GMT
• Title: Retriever: Composing Closed-Loop Asynchronous Robot Programs
• Abstract summary: Retriever is a graph of stateful causal stream functions executed on explicit run clocks.<n>It compiles these streams into a runtime that supports multiple backends.<n>We evaluate Retriever through a real-robot case study.
• Score: 30.120352978306574
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Building long-horizon robot agents requires composing closed-loop pipelines -- perception, belief update, planning, and control -- whose components run at different clocks and with variable latency. Today, these systems are often assembled with ad-hoc concurrency and pub/sub conventions that make timing and input-consumption semantics implicit, yielding schedule-dependent behavior that is hard to reproduce, debug, and reuse. Current solutions typically solve parts of this problem at either the algorithmic or the systems layer, but not both. In this work, we propose Retriever, which spans the entire stack: an asynchronous decision model, a programming model, a runtime, and an example closed-loop agent pipeline. Retriever represents an agent as a graph of stateful causal stream functions executed on explicit run clocks. We formalize this view via an asynchronous environment-agent loop over continuous-time streams and show that finite-memory causal policies can be represented by compositions of these operators. Retriever compiles these graphs into a runtime that supports multiple backends, enabling systematic debugging across running environments and deterministic replay from logged asynchronous data. We evaluate Retriever through a real-robot case study together with controlled studies of runtime overhead and deterministic replay behavior.

Related papers

• Inference-Time Graph Engineering for Multi-Agent LLM Workflows [51.56038180639833]
  We introduce ReActNet, a training-free framework that compiles a query and a set of role-specialized agents into a sequence of directed communication graphs.<n>We show that ReActNet consistently improves over fixed-topology and learned-topology baselines while maintaining competitive inference cost.
  arXiv  Detail & Related papers  (2026-09-04T23:32:11Z)
• PhyAgentOS: A Self-Evolving Operating System for Embodied Agents with Decoupled Cognitive Planning and Physical Execution [49.776611937968]
  We present PhyAgentOS, a foundation delivering scheduling, verification, memory, benchmarking, and safety as system-level services.<n>Its Session-Centered treats a session, not an action, as the minimum unit of scheduling, compatibility, supervised execution, evidence collection, and acceptance.<n>SessionVerifier distinguishes execution termination from semantic task completion via evidence-grounded verdicts of success, failure, or replan.<n> Benchmarking reuses the deployment session and verification path, so results trace to real execution.
  arXiv  Detail & Related papers  (2026-07-18T04:46:53Z)
• MemOps: Benchmarking Lifecycle Memory Operations in Long-Horizon Conversations [50.24315431387575]
  We introduce MemOps, a benchmark that reformulates conversational memory as a sequence of lifecycle operations.<n>MemOps represents each memory event with a structured trace specifying its trigger, target, scope, state transition, and supporting evidence.<n>Across long-context, retrieval-based, parametric and managed-memory systems, MemOps disentangles failure modes that final-answer accuracy alone conceals.
  arXiv  Detail & Related papers  (2026-07-14T15:33:44Z)
• Toward the Right Analytical Model and System Software for Autonomous Driving Systems: Open Problems and Research Directions [0.0]
  This paper surveys the remaining gaps along five dimensions: units of timing constraints, timing metrics, resource models, execution-time variability, and safety integration.<n>This raises a two-sided question: what analytical models are needed to reason about timing in AD systems, and what system software is needed to realize, observe, and enforce those models on real platforms.
  arXiv  Detail & Related papers  (2026-07-05T06:10:47Z)
• OpenRath: Session-Centered Runtime State for Agent Systems [2.9568259440852]
  OpenRath is a PyTorch-like programming model for multi-agent, multi-session systems.<n>Session is branchable, inspectable, replayable, backend-aware, and composable.<n>Report presents the programming model, architecture, audited milestones, and evidence protocol.
  arXiv  Detail & Related papers  (2026-06-17T15:19:19Z)
• Multi-Agent Computer Use [72.79887808312706]
  We argue that we should move towards evaluating and building multi-agent computer use (MACU) systems.<n>We propose a general multi-agent setup in which a manager model decomposes computer use tasks as a directed acyclic graph (DAG)<n>At each iteration, the manager dispatches parallel CUA subagents to carry out nodes on the ready frontier of the DAG.
  arXiv  Detail & Related papers  (2026-06-01T01:29:36Z)
• The Log is the Agent: Event-Sourced Reactive Graphs for Auditable, Forkable Agentic Systems [0.0]
  We describe ActiveGraph, a runtime that inverts the typical agent framework.<n>The append-only event log is the source of truth; the working graph is a deterministic projection of that log.<n>No component instructs another; coordination happens entirely through the shared graph.
  arXiv  Detail & Related papers  (2026-05-21T04:55:38Z)
• A Physical Agentic Loop for Language-Guided Grasping with Execution-State Monitoring [5.437966695589128]
  We reformulate language-guided grasping as a bounded embodied agent operating over grounded execution states.<n>We introduce a physical agentic loop that wraps an unmodified learned manipulation primitive.<n>We validate the resulting loop on a mobile manipulator with an eye-in-hand D405 camera.
  arXiv  Detail & Related papers  (2026-04-08T08:01:35Z)
• AMA: Adaptive Memory via Multi-Agent Collaboration [54.490349689939166]
  We propose Adaptive Memory via Multi-Agent Collaboration (AMA), a novel framework that leverages coordinated agents to manage memory across multiple granularities.<n>AMA significantly outperforms state-of-the-art baselines while reducing token consumption by approximately 80% compared to full-context methods.
  arXiv  Detail & Related papers  (2026-01-28T08:09:49Z)
• Dispider: Enabling Video LLMs with Active Real-Time Interaction via Disentangled Perception, Decision, and Reaction [81.34648970317383]
  We present Dispider, a system that disentangles Perception, Decision, and Reaction.<n>Experiments show that Dispider not only maintains strong performance in conventional video QA tasks, but also significantly surpasses previous online models in streaming scenario responses.
  arXiv  Detail & Related papers  (2025-01-06T18:55:10Z)
• Tooling Offline Runtime Verification against Interaction Models : recognizing sliced behaviors using parameterized simulation [0.4199844472131921]
  offline runtime verification involves the static analysis of executions of a system against a specification. For distributed systems, it is generally not possible to characterize executions in the form of global traces, given the absence of a global clock. We propose an algorithm that verifies the conformity of such traces against formal specifications called Interactions.
  arXiv  Detail & Related papers  (2024-03-05T16:09:55Z)
• TimeGraphs: Graph-based Temporal Reasoning [64.18083371645956]
  TimeGraphs is a novel approach that characterizes dynamic interactions as a hierarchical temporal graph. Our approach models the interactions using a compact graph-based representation, enabling adaptive reasoning across diverse time scales. We evaluate TimeGraphs on multiple datasets with complex, dynamic agent interactions, including a football simulator, the Resistance game, and the MOMA human activity dataset.
  arXiv  Detail & Related papers  (2024-01-06T06:26:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.