Treating Run-time Execution History as a First-Class Citizen: Co-Versioning Run-time Behavior alongside Code

• URL: http://arxiv.org/abs/2604.16933v1
• Date: Sat, 18 Apr 2026 09:36:12 GMT
• Title: Treating Run-time Execution History as a First-Class Citizen: Co-Versioning Run-time Behavior alongside Code
• Abstract summary: This paper argues that this mismatch contributes to a blind spot in software evolution analysis and CI.<n>We propose textitBehavioral Co-Versioning, a paradigm that couples the Git history with a textitBehavioral Archive.<n>This enables semantic diffing, behavior-aware regression localization, and retrospective auditing by querying historical executions.
• Score: 2.6397379133308214
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Behavioral Co-Versioning remains absent from mainstream practice: while developers routinely version source code with Git, they rarely persist and query how run-time behavior evolves across revisions. This paper argues that this mismatch contributes to a blind spot in software evolution analysis and CI, where rich execution information is discarded and typically reduced to pass/fail outcomes -- despite partial test oracles, flakiness, and silent output or performance drift. We propose \textit{Behavioral Co-Versioning}, a paradigm that couples the Git history with a \textit{Behavioral Archive}: an append-only, queryable store of selected run-time observations (e.g., method I/O and performance signals) collected during test runs and keyed by commit and test context. This enables semantic diffing, behavior-aware regression localization, and retrospective auditing by querying historical executions, complementing proactive, signal-specific monitoring tools. We first outline a minimal data model and change diagnostics based on code/test/behavior fingerprints, and then demonstrate feasibility with a laptop-scale prototype that replays historical commits of a Python project, archives run-time observations in a local Parquet-backed store, and detects behavioral changes not apparent from textual diffs.

Related papers

• Dr. Claw: An AI Scientist Workspace for Vibe Research [64.18459471289147]
  Dr. Claw is an open-source workspace that wraps existing coding-agent executors in a controllable human-in-the-loop workflow.<n> Persistent state objects, a reusable skill library, and multi-executor coordination link human decisions to AI execution.<n>Dr. Claw scores higher on research completeness while persisting an auditable, recoverable process trail.
  arXiv  Detail & Related papers  (2026-08-31T20:58:47Z)
• ScrambleToolBench: Agents Search Exhaustively Even When Their Own Map Points to the Next Step [74.83535434786145]
  We introduce ScrambleToolBench, an interactive terminal benchmark designed to isolate behavioral reasoning.<n>The benchmark requires agents to uncover hidden tool behaviors entirely through trial-and-error interaction.<n>Our evaluation of state-of-the-art language models reveals that successful initial discovery does not translate into robust adaptation.
  arXiv  Detail & Related papers  (2026-08-03T15:07:46Z)
• PhyAgentOS: A Self-Evolving Operating System for Embodied Agents with Decoupled Cognitive Planning and Physical Execution [49.776611937968]
  We present PhyAgentOS, a foundation delivering scheduling, verification, memory, benchmarking, and safety as system-level services.<n>Its Session-Centered treats a session, not an action, as the minimum unit of scheduling, compatibility, supervised execution, evidence collection, and acceptance.<n>SessionVerifier distinguishes execution termination from semantic task completion via evidence-grounded verdicts of success, failure, or replan.<n> Benchmarking reuses the deployment session and verification path, so results trace to real execution.
  arXiv  Detail & Related papers  (2026-07-18T04:46:53Z)
• SemaDiff: Identifying Semantic-Changing Commits with Generated Code and Tests [5.3138104073306005]
  Distinguishing semantic-preserving commits from changing ones remains an open challenge in software repository mining.<n>SemaDiff is a novel approach for identifying semantic-preserving commits through comparison of similar test execution on pre- and post-commit versions.<n>SemaDiff distinguishes accurately semantic-preserving from -- changing commits in about 76% of the cases, with a 100% precision in semantic-changing commit detection.
  arXiv  Detail & Related papers  (2026-07-14T12:23:22Z)
• TTHE: Test-Time Harness Evolution [50.26245555541721]
  Existing approaches optimize such harnesses before deployment, searching training or development data for a fixed agent workflow that is then frozen at test time.<n>We ask whether the harness can instead be optimized during evaluation itself, using only the unlabeled execution traces the agent produces on the test inputs.<n>During evaluation, TTHE maintains a population of candidate harnesses and refines them through an agentic proposer that reasons over their execution traces.<n>A judge then commits an improved harness from execution-derived proxy signals, and the selected program persists to govern subsequent inputs.
  arXiv  Detail & Related papers  (2026-07-09T05:53:39Z)
• What Resolve Rate Hides: Trajectory Structure Diagnostics for Coding Agents [14.988334585821939]
  Coding agents are ranked almost entirely by resolve rate.<n>A single pass/fail label says nothing about why a run failed or why an accepted run spent extra steps, time, or tokens.<n>We present TraceProbe, a trajectory-diagnostic framework that recovers what resolve rate hides.
  arXiv  Detail & Related papers  (2026-07-07T12:09:46Z)
• A History-Aware Visually Grounded Critic for Computer Use Agents [70.50900668601969]
  HiViG is a test-time framework built around a multimodal critic trained on real GUI trajectories.<n>At test time, HiViG integrates the critic into the policy decision loop to provide macro-action history.<n>Visually grounded critique verifies raw execution coordinates against the current screenshot to intercept errors before execution.
  arXiv  Detail & Related papers  (2026-06-09T16:39:10Z)
• ClawMark: A Living-World Benchmark for Multi-Turn, Multi-Day, Multimodal Coworker Agents [77.22389710754452]
  We introduce bench, a benchmark for coworker agents built around multi-turn multi-day tasks.<n>The current release contains 100 tasks across 13 professional scenarios, executed against five stateful sandboxed services.<n>The strongest model reaches 75.8 weighted score, but the best strict Task Success is only 20.0%, indicating that partial progress is common.
  arXiv  Detail & Related papers  (2026-04-26T16:05:02Z)
• Interactive Tracking: A Human-in-the-Loop Paradigm with Memory-Augmented Adaptation [60.96010213186819]
  We introduce Interactive Tracking, a new paradigm that allows users to guide the tracker at any time using natural language commands.<n>We present InteractTrack, the first large-scale benchmark for interactive tracking, containing 150 videos with dense bounding box annotations and timestamped language instructions.<n>Third, we introduce Interactive Memory-Augmented Tracking (IMAT), a new baseline that employs a dynamic memory mechanism to learn from user feedback and update tracking behavior accordingly.
  arXiv  Detail & Related papers  (2026-04-02T12:33:27Z)
• AlgoVeri: An Aligned Benchmark for Verified Code Generation on Classical Algorithms [54.99368693313797]
  Existing benchmarks test only individual languages/tools, so the performance numbers are not directly comparable.<n>We address this gap with AlgoVeri, a benchmark that evaluates vericoding of $77$ classical algorithms in Dafny, Verus, and Lean.
  arXiv  Detail & Related papers  (2026-02-10T06:58:26Z)
• Towards Observation Lakehouses: Living, Interactive Archives of Software Behavior [2.6397379133308214]
  In prior work, we addressed representation with Sequence Sheets, StimulusResponse Matrices, and StimulusResponse Cubes.<n>In this paper, we introduce observation lakehouses that operationalize continual SRCs.<n>We ingest data from controlled pipelines (SOLAS) and CILAS, enabling n-version assessment, behavioral clustering, and consensus oracles.
  arXiv  Detail & Related papers  (2025-12-02T14:12:36Z)
• SwingArena: Competitive Programming Arena for Long-context GitHub Issue Solving [90.32201622392137]
  We present SwingArena, a competitive evaluation framework for Large Language Models (LLMs)<n>Unlike traditional static benchmarks, SwingArena models the collaborative process of software by pairing LLMs as iterations, who generate patches, and reviewers, who create test cases and verify the patches through continuous integration (CI) pipelines.
  arXiv  Detail & Related papers  (2025-05-29T18:28:02Z)
• A Case Study on Model Checking and Runtime Verification for Awkernel [0.7199733380797578]
  It is difficult for humans to manually review concurrent behaviors or to write test cases covering all possible executions.<n>This paper proposes a development method for concurrent software, such as schedulers.
  arXiv  Detail & Related papers  (2025-03-12T11:27:45Z)
• GitChameleon: Unmasking the Version-Switching Capabilities of Code Generation Models [16.6780665807022]
  textbfGitChameleon is a novel, manually curated dataset comprising 116 Python code completion problems. GitChameleon is designed to rigorously assess the ability of modern large language models to generate version-specific code.
  arXiv  Detail & Related papers  (2024-11-05T23:34:06Z)
• Multiversion Hindsight Logging for Continuous Training [8.861816325333963]
  Multiversion Hindsight Logging allows engineers to query past versions, even when older versions logged different data. FlorDB presents a unified relational model for efficient handling of historical queries.
  arXiv  Detail & Related papers  (2023-10-11T21:10:15Z)
• Autoregressive Search Engines: Generating Substrings as Document Identifiers [53.0729058170278]
  Autoregressive language models are emerging as the de-facto standard for generating answers. Previous work has explored ways to partition the search space into hierarchical structures. In this work we propose an alternative that doesn't force any structure in the search space: using all ngrams in a passage as its possible identifiers.
  arXiv  Detail & Related papers  (2022-04-22T10:45:01Z)
• Self-Supervised Log Parsing [59.04636530383049]
  Large-scale software systems generate massive volumes of semi-structured log records. Existing approaches rely on log-specifics or manual rule extraction. We propose NuLog that utilizes a self-supervised learning model and formulates the parsing task as masked language modeling.
  arXiv  Detail & Related papers  (2020-03-17T19:25:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.