APEX-EM: Non-Parametric Online Learning for Autonomous Agents via Structured Procedural-Episodic Experience Replay

• URL: http://arxiv.org/abs/2603.29093v2
• Date: Thu, 02 Apr 2026 21:09:27 GMT
• Title: APEX-EM: Non-Parametric Online Learning for Autonomous Agents via Structured Procedural-Episodic Experience Replay
• Abstract summary: LLM-based autonomous agents lack persistent procedural memory.<n>We present APEX-EM, a non-parametric online learning framework that accumulates, retrieves, and reuses structured procedural plans.
• Score: 7.370176470430802
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: LLM-based autonomous agents lack persistent procedural memory: they re-derive solutions from scratch even when structurally identical tasks have been solved before. We present APEX-EM, a non-parametric online learning framework that accumulates, retrieves, and reuses structured procedural plans without modifying model weights. APEX-EM introduces: (1) a structured experience representation encoding the full procedural-episodic trace of each execution -- planning steps, artifacts, iteration history with error analysis, and quality scores; (2) a Plan-Retrieve-Generate-Iterate-Ingest (PRGII) workflow with Task Verifiers providing multi-dimensional reward signals; and (3) a dual-outcome Experience Memory with hybrid retrieval combining semantic search, structural signature matching, and plan DAG traversal -- enabling cross-domain transfer between tasks sharing no lexical overlap but analogous operational structure. Successful experiences serve as positive in-context examples; failures as negative examples with structured error annotations. We evaluate on BigCodeBench, KGQAGen-10k, and Humanity's Last Exam using Claude Sonnet 4.5 and Opus 4.5. On KGQAGen-10k, APEX-EM achieves 89.6% accuracy versus 41.3% without memory (+48.3pp), surpassing the oracle-retrieval upper bound (84.9%). On BigCodeBench, it reaches 83.3% SR from a 53.9% baseline (+29.4pp), exceeding MemRL's +11.0pp gain under comparable frozen-backbone conditions (noting backbone differences controlled for in our analysis). On HLE, entity graph retrieval reaches 48.0% from 25.2% (+22.8pp). Ablations show component value is task-dependent: rich judge feedback is negligible for code generation but critical for structured queries (+10.3pp), while binary-signal iteration partially compensates for weaker feedback.

Related papers

• A-SR: Self-Evolving Agentic LLMs for Symbolic Regression via Hierarchical Coordination [51.06539604709775]
  We propose A-SR, a self-evolving agentic framework that shifts the control unit from expression edits to role-conditioned evidence views.<n>A-SR coordinates formula discovery through routing among coordination protocols, an online evaluator-reward role policy, and state-routed process memory.
  arXiv  Detail & Related papers  (2026-08-05T14:01:10Z)
• Fantastic Adaptive Taxonomies and How to Use Them [100.20614173157708]
  AdaMAST converts execution traces into a compact, evidence-grounded failure taxonomy.<n>No code is hand-authored, no trace human-annotated.<n>In agent-system search, taxonomy-coded diagnoses of failed candidates outperform free-form.
  arXiv  Detail & Related papers  (2026-07-17T17:41:16Z)
• Cost-Effective Agent Harnesses for Abstract Reasoning and Generalization on ARC-AGI-1 [4.486629113012585]
  Recent progress on ARC-AGI-1 has come from two regimes: heavy test-time compute over frontier models, or benchmark-specific training in which small models are fine-tuned on ARC data.<n>We study what is recoverable through architecture alone, building agentic harnesses that decompose pattern-discovery and program-synthesis stages explicitly.
  arXiv  Detail & Related papers  (2026-07-07T19:49:35Z)
• When the Database Fails: Prompting LLM Dialogue Agents for Safe Recovery in Task-Oriented Dialogue [1.9116784879310027]
  We study a lightweight prompting-based recovery approach that improves robustness without retraining or additional model calls.<n>We compare three response strategies, including a guided recovery prompt conditioned on structured database status.<n>We find that naive agents hallucinate on 30.5% of failure turns on MultiWOZ and 20.9% on SGD.
  arXiv  Detail & Related papers  (2026-06-30T08:18:11Z)
• Efficient Visual Pointing for Embodied AI:Agent-Driven Data Synthesis, Cross-Block Attention, and Iterative Correction [55.11480729304395]
  We describe our PointArena 2026 solution, which achieves 77.2% overall accuracy and ranks second on the benchmark.<n>The ap proach targets three failure modes. First, agent-driven syn thesis builds large semantic and anchor-relative candidate pools.<n>Second, a determinis tic steerable-data pipeline creates a verified 10,000-sample main set, plus reserve samples, using masks, templates, and path verification.
  arXiv  Detail & Related papers  (2026-06-29T06:39:03Z)
• InvestPhilBench: A Multi-Layer Dynamic Benchmark for Evaluating Large Language Model Procedural Reasoning in Expert Investment Philosophy [0.9924185669370708]
  InvestPhilBench is a dynamic benchmark spanning eight cognitive tiers.<n>The v0.6 release comprises 118 primary-source-verified investment principle cards.<n>For reproducible scoring at scale we introduce the Benchmark Automated Scoring Pipeline.
  arXiv  Detail & Related papers  (2026-06-24T15:53:20Z)
• Plan, Watch, Recover: A Benchmark and Architectures for Proactive Procedural Assistance [49.46086226939232]
  We release textbfEgoProactive, a wearable-egocentric dataset for proactive procedural assistance with explicit Out-of-Plan (OOP) annotations and recovery steps.<n>We propose a textbfdecoupled planner--interaction architecture specialized for procedural state, visual cues, and recovery injection.<n>In extensive experiments, our trained Llama-4 system substantially improves objective intervention quality over strong proprietary baselines.
  arXiv  Detail & Related papers  (2026-06-03T14:52:03Z)
• Converted, Not Equivalent: Benchmarking Codebase Conversion via Observational Equivalence [56.25095230687242]
  Coding agents often over-trust their own local validation routines and declare success on artifacts that satisfy surface checks.<n>This problem is especially acute in conversion, where prior evaluation is outcome-driven.<n>Across blind conversion attempts, the best system reaches 26.7--28.9% overall pass rate despite Spec pass rates up to 91.1%.<n>This suggests that failures stem more from contract-misaligned self-validation than from limited budget or backbone strength.
  arXiv  Detail & Related papers  (2026-05-27T19:57:15Z)
• Adaptive ToR: Complexity-Aware Tree-Based Retrieval for Pareto-Optimal Multi-Intent NLU [0.15293427903448018]
  Multi-intent natural language understanding requires retrieval systems that simultaneously achieve high accuracy and computational efficiency.<n>This paper proposes Adaptive Tree-of-Retrieval (Adaptive ToR), a complexity-aware retrieval architecture that dynamically configures retrieval topology based on query characteristics.
  arXiv  Detail & Related papers  (2026-04-27T09:24:10Z)
• Training LLMs for Multi-Step Tool Orchestration with Constrained Data Synthesis and Graduated Rewards [76.49428173793386]
  LLMs must invoke multiple dependent APIs in the correct order while propagating intermediate outputs.<n>Training models to handle such faces two obstacles: existing environments focus on simple per-turn function calls with simulated data, and binary rewards.<n>We present a framework addressing both challenges. First, we construct a reinforcement learning environment backed by a large-scale cache of real API responses, enabling a data synthesis pipeline that samples valid multi-step orchestration traces.<n>Second, we propose a graduated reward design that decomposes correctness into atomic validity and orchestration.
  arXiv  Detail & Related papers  (2026-03-25T18:31:39Z)
• TopoChunker: Topology-Aware Agentic Document Chunking Framework [5.304983617085637]
  TopoChunker is an agentic framework that maps heterogeneous documents onto a Structured Intermediate Representation (SIR)<n>To balance structural fidelity with computational cost, TopoChunker employs a dual-agent architecture.<n>It outperforms the strongest LLM-based baseline by 8.0% in absolute generation accuracy and achieves an 83.26% Recall@3, while simultaneously reducing token overhead by 23.5%.
  arXiv  Detail & Related papers  (2026-03-19T02:15:10Z)
• Graph-Native Cognitive Memory for AI Agents: Formal Belief Revision Semantics for Versioned Memory Architectures [0.6091702876917279]
  Kumiho is a graph-native cognitive memory architecture grounded in formal belief revision semantics.<n>The architecture implements a dual-store model (Redis working memory, Neo4j long-term graph) with hybrid fulltext and vector retrieval.
  arXiv  Detail & Related papers  (2026-03-18T00:59:49Z)
• ACAR: Adaptive Complexity Routing for Multi-Model Ensembles with Auditable Decision Traces [3.151184728006369]
  We present ACAR, a measurement framework for studying multi-model orchestration under auditable conditions.<n>ACAR uses self-consistency variance (sigma) computed from N=3 probe samples to route tasks across single-model, two-model, and three-model execution modes.<n>We evaluate ACAR on 1,510 tasks spanning four benchmarks, producing more than 7,550 auditable runs.
  arXiv  Detail & Related papers  (2026-02-06T23:27:17Z)
• Solver-in-the-Loop: MDP-Based Benchmarks for Self-Correction and Behavioral Rationality in Operations Research [19.31559944205485]
  Operations Research practitioners routinely debug infeasible models through an iterative process.<n>We introduce two benchmarks that place the textbfsolver in the evaluation loop<n>We find that domain-specific RLVR training enables an 8B model to surpass frontier APIs.
  arXiv  Detail & Related papers  (2026-01-28T20:02:44Z)
• Eigen-1: Adaptive Multi-Agent Refinement with Monitor-Based RAG for Scientific Reasoning [53.45095336430027]
  We develop a unified framework that combines implicit retrieval and structured collaboration.<n>On Humanity's Last Exam (HLE) Bio/Chem Gold, our framework achieves 48.3% accuracy.<n>Results on SuperGPQA and TRQA confirm robustness across domains.
  arXiv  Detail & Related papers  (2025-09-25T14:05:55Z)
• Evaluating the Use of LLMs for Documentation to Code Traceability [3.076436880934678]
  Large Language Models can establish trace links between various software documentation and source code.<n>We create two novel datasets from two open-source projects (Unity Catalog and Crawl4AI)<n>Results show that the best-performing LLM achieves F1-scores of 79.4% and 80.4% across the two datasets.
  arXiv  Detail & Related papers  (2025-06-19T16:18:53Z)
• ProofAug: Efficient Neural Theorem Proving via Fine-grained Proof Structure Analysis [50.020850767257095]
  We propose ProofAug, a procedure that equips LLMs with automation methods at various granularities.<n>Our method is validated on the miniF2F benchmark using the open-source deep-math-7b-base model and the Isabelle proof assistant.<n>We also implement a Lean 4 version of ProofAug that can improve the pass@1 performance of Kimina-Prover-seek-Distill-1.5B from 44.3% to 50.4%.
  arXiv  Detail & Related papers  (2025-01-30T12:37:06Z)
• ALoRE: Efficient Visual Adaptation via Aggregating Low Rank Experts [71.91042186338163]
  ALoRE is a novel PETL method that reuses the hypercomplex parameterized space constructed by Kronecker product to Aggregate Low Rank Experts.<n>Thanks to the artful design, ALoRE maintains negligible extra parameters and can be effortlessly merged into the frozen backbone.
  arXiv  Detail & Related papers  (2024-12-11T12:31:30Z)
• Integral Continual Learning Along the Tangent Vector Field of Tasks [112.02761912526734]
  We propose a lightweight continual learning method which incorporates information from specialized datasets incrementally. It maintains a small fixed-size memory buffer, as low as 0.4% of the source datasets, which is updated by simple resampling. Our method achieves strong performance across various buffer sizes for different datasets.
  arXiv  Detail & Related papers  (2022-11-23T16:49:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.