Persistent AI Agents in Academic Research: A Single-Investigator Implementation Case Study

• URL: http://arxiv.org/abs/2605.26870v1
• Date: Tue, 26 May 2026 11:28:36 GMT
• Title: Persistent AI Agents in Academic Research: A Single-Investigator Implementation Case Study
• Abstract summary: The unit of analysis was the persistent human-agent environment: researcher, agent, memory layer, tools, repositories, scheduled jobs, specialized agent roles, and governance rules.<n>A strict May 2026 trajectory subset captured 627 model-related events and 73.95 million recorded tokens, of which 82.9% were cache reads.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Background: Large language models are typically evaluated as models, benchmarks, or short conversational episodes. Less is known about what happens when an agent is embedded persistently in a real academic research environment with durable memory, local files, external tools, scheduled routines, delegated roles, and explicit safety protocols. Methods: A structured self-observed implementation case study was conducted from January 31 to May 25, 2026. The unit of analysis was the persistent human-agent environment: researcher, agent runtime, memory layer, tools, repositories, scheduled jobs, specialized agent roles, and governance rules. Outcomes were organized using PARE-M (Persistent Agentic Research Environment Measurement), a measurement framework covering architecture, utilization, artifact production, resource use, reproducibility, and governance. Results: Recoverable main-agent telemetry contained 75,671 de-duplicated records across 96 active days, with 8,059 user-role and 23,710 assistant-role messages. The workspace included 502 memory-related files, 17 configured agent directories, and 57 skill files. Active system time was 579.7 hours (30-minute capped-gap estimate). Memory-derived records identified 482 output-proxy events and 889 failure, verification, correction, or protocol-proxy events. A strict May 2026 trajectory subset captured 627 model-completed events and 73.95 million recorded tokens, of which 82.9% were cache reads. Conclusions: The workflow was cache-dominant, suggesting that persistent agentic environments may shift the economic unit from cost per token to cost per completed artifact. Future evaluations should use artifact-level denominators, reproducible parsing rules, correction taxonomies, and independent coding of governance events.

Related papers

• Cost-Effective Repository Exploration for Agentic Issue Localization [0.2715382584771446]
  We study whether this stage can be delegated to lower-cost models.<n>We measure early candidate discovery, top-three gold-file coverage, strict file-set recovery, agent time, and token usage.<n>Highest-quality explorer leads across localization metrics, but substantially cheaper operating points emerge.
  arXiv  Detail & Related papers  (2026-08-30T09:16:33Z)
• ScrambleToolBench: Agents Search Exhaustively Even When Their Own Map Points to the Next Step [74.83535434786145]
  We introduce ScrambleToolBench, an interactive terminal benchmark designed to isolate behavioral reasoning.<n>The benchmark requires agents to uncover hidden tool behaviors entirely through trial-and-error interaction.<n>Our evaluation of state-of-the-art language models reveals that successful initial discovery does not translate into robust adaptation.
  arXiv  Detail & Related papers  (2026-08-03T15:07:46Z)
• Messier: A High-Resolution Corpus for Cross-Benchmark Agent Evaluation [37.16322609017877]
  Messier is a unified corpus of 957,253 records that span 30 benchmarks, 714 agents, 11,891 tasks, and 74,205 verifiers.<n>M Messier consolidates public benchmark scores and supplements them with five-agent runs across six underrepresented professional and scientific domains.
  arXiv  Detail & Related papers  (2026-07-28T15:50:19Z)
• AGORA: An Archive-Grounded Benchmark for Agentic Workplace Document Reasoning [52.23991730630292]
  Large language models are increasingly deployed as agents that reason over documents rather than answer from parametric knowledge.<n>We study archive-grounded reasoning: locating sparse evidence across a large, messy collection of workplace files, reconciling inconsistent terminology, units, and time conventions, and computing an answer.<n>We introduce Agora, a benchmark pairing 362 questions with eight domain collections of 9,664 authentic documents and 372M tokens, far exceeding any model's context window.
  arXiv  Detail & Related papers  (2026-06-23T12:57:18Z)
• Handoff Debt: The Rediscovery Cost When Coding Agents Take Over Interrupted Tasks [0.0]
  Real software work is messier because tasks are interrupted, reassigned, reviewed, and resumed from partial states left by another agent or engineer.<n>We study this missing dimension through emphhandoff debt: the rediscovery cost imposed when a predecessor's work is opaque or incomplete.<n>Our takeover protocol interrupts a coding agent at deterministic handoff points, freezes the repository, and evaluates successor agents under four handoff views.
  arXiv  Detail & Related papers  (2026-06-01T20:40:38Z)
• Workspace-Bench 1.0: Benchmarking AI Agents on Workspace Tasks with Large-Scale File Dependencies [29.138677906501865]
  We introduce Workspace-Bench, a benchmark for evaluating AI agents on Workspace Learning involving Large-Scale File Dependencies.<n>We construct realistic workspaces with 5 worker profiles, 74 file types, 20,476 files (up to 20GB) and curate 388 tasks, each with its own file dependency graph, evaluated across 7,399 total rubrics.
  arXiv  Detail & Related papers  (2026-05-05T10:17:06Z)
• Claw-Eval-Live: A Live Agent Benchmark for Evolving Real-World Workflows [67.92316850084575]
  We introduce Claw-Eval-Live, a live benchmark for workflow agents.<n>Each release is constructed from public workflow-demand signals.<n>Claw-Eval-Live records execution traces, audit logs, service state, and post-run workspace artifacts.
  arXiv  Detail & Related papers  (2026-04-30T17:23:19Z)
• The Last Human-Written Paper: Agent-Native Research Artifacts [106.47848184955576]
  We introduce the Agent-Native Research Artifact (ARA), a protocol that replaces the narrative paper with a machine-executable research package.<n>Three mechanisms support the ecosystem: a Live Research Manager that captures decisions and dead ends during ordinary development; an ARA Compiler that translates legacy PDFs and repos into ARAs; and an ARA-native review system that automates objective checks so human reviewers can focus on significance, novelty, and taste.
  arXiv  Detail & Related papers  (2026-04-27T16:23:09Z)
• Exploring Robust Multi-Agent Workflows for Environmental Data Management [2.556033550563853]
  EnviSmart is a production data management system deployed on campus-wide storage infrastructure for environmental research.<n>It externalizes behaviors (governance constraints), domain knowledge (retrievable context), and skills (tool-using procedures) as persistent, interlocking artifacts.
  arXiv  Detail & Related papers  (2026-04-02T05:46:40Z)
• OmniEAR: Benchmarking Agent Reasoning in Embodied Tasks [52.87238755666243]
  We present OmniEAR, a framework for evaluating how language models reason about physical interactions, tool usage, and multi-agent coordination in embodied tasks.<n>We model continuous physical properties and complex spatial relationships across 1,500 scenarios spanning household and industrial domains.<n>Our systematic evaluation reveals severe performance degradation when models must reason from constraints.
  arXiv  Detail & Related papers  (2025-08-07T17:54:15Z)
• ThinkGeo: Evaluating Tool-Augmented Agents for Remote Sensing Tasks [64.86209459039313]
  ThinkGeo is an agentic benchmark designed to evaluate tool-augmented agents on remote sensing tasks via structured tool use and multi-step planning.<n>We implement a ReAct-style interaction loop and evaluate both open and closed-source LLMs on 486 structured agentic tasks with 1,773 expert-verified reasoning steps.<n>Our analysis reveals notable disparities in tool accuracy and planning consistency across models.
  arXiv  Detail & Related papers  (2025-05-29T17:59:38Z)
• FieldWorkArena: Agentic AI Benchmark for Real Field Work Tasks [52.47895046206854]
  FieldWorkArena is a benchmark for agentic AI targeting real-world field work.<n>This paper defines a new action space that agentic AI should possess for real world work environment benchmarks.
  arXiv  Detail & Related papers  (2025-05-26T08:21:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.