CrossAudit: A Git-Native, Cross-Vendor Audit Loop for Agentic Science

• URL: http://arxiv.org/abs/2608.28631v1
• Date: Wed, 05 Aug 2026 12:35:59 GMT
• Title: CrossAudit: A Git-Native, Cross-Vendor Audit Loop for Agentic Science
• Abstract summary: CrossAudit is a protocol for supervising autonomous research pipelines.<n>Each increment of work is audited by an agent from a different vendor against a rulebook a human wrote and versioned.<n>Reports, verdicts, disputes and rulings are git commits, so the supervision history can be re-read and cited.
• Score: 5.610929955077264
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: An AI scientist should not grade its own homework. Yet in the systems we examined, the agent that reviews the work usually comes from the same model family as the agent that produced it, or at least from the same vendor. Model evaluators are known to favour their own generations. Whether models trained alike also share blind spots is a conjecture, not a settled finding, but if they do, the reviewer inherits the author's. The record of what was flagged and what was waved through often sits in platform logs that nobody outside can replay. We present CrossAudit, a protocol for supervising autonomous research pipelines. It rests on three commitments. Each increment of work is audited by an agent from a different vendor against a rulebook a human wrote and versioned. Reports, verdicts, disputes and rulings are git commits, so the supervision history can be re-read and cited; raw model exchanges are not yet part of that record. Scripted checks run before any model does. Advisory judgement never gates the pipeline: a model blocks only by citing a rule, and no model may waive a deterministic failure. Blockers that survive a bounded number of revision rounds go to a person. We state the protocol as eight invariants. We describe a reference implementation built from GitHub Actions and a few hundred lines of Python, and report a live deployment of a closely related variant in a computational-chemistry pipeline. We also ran a seeded-defect trial (30 increments, 43 seeded defects, one run per configuration). A cross-vendor audit of our own repository then voided its blinding. We adopt that audit's findings and report the corrected results. The trial shows that two vendors read the same rulebook differently. It does not show that either is better. The strongest evidence here is the committed, uncontrolled record of cross-vendor audits of this paper itself.

Related papers

• Dr. Claw: An AI Scientist Workspace for Vibe Research [64.18459471289147]
  Dr. Claw is an open-source workspace that wraps existing coding-agent executors in a controllable human-in-the-loop workflow.<n> Persistent state objects, a reusable skill library, and multi-executor coordination link human decisions to AI execution.<n>Dr. Claw scores higher on research completeness while persisting an auditable, recoverable process trail.
  arXiv  Detail & Related papers  (2026-08-31T20:58:47Z)
• Where Accountability Lives: Mapping Human Responsibility to Workflow Artifacts in Agentic Software Development [0.0]
  Coding agents author, open pull requests, and push code in production repositories.<n>We read both against the workflow events that leave artifacts, across four agentic coding tools and eighteen governing policy documents from seven providers.<n>One provider bars the developer who assigned a task from approving the resulting pull request; another documents an agent that approves pull requests below a configured risk threshold and can dismiss reviews.<n>We replace the usual three-way distinction between enforced, advisory and absent verification with a grid separating whether a mechanism compels the check from who performs it.
  arXiv  Detail & Related papers  (2026-08-16T11:11:04Z)
• Reconcile Once, Write Anytime: A Trust-Tiered Librarian and a Multi-Agent Writer for Drift-Free, Point-in-Time Research [7.426963307811556]
  We present a two-tier agentic system that separates a maintained, point-in-time knowledge library from report writing.<n>A portable multi-agent "writer" runtime composes a contradiction-free, evidence-grounded report at any knowledge cutoff T.<n>We evaluate on a self-collected, public corpus of 6,130 sources yielding 555,926 evidence cards.
  arXiv  Detail & Related papers  (2026-08-13T09:09:28Z)
• Tencent WorkBuddy Bench: A Multi-Domain Coding-Agent Benchmark with Contamination-Resistant Task Construction [57.138342889101345]
  Tencent WorkBuddy Bench is a multi-domain evaluation suite for coding agents.<n>This report documents its construction methodology, scoring protocol, and a cross-model leaderboard.
  arXiv  Detail & Related papers  (2026-07-23T04:34:06Z)
• Abliteration Is Not a Scalpel: Off-Target Effects of Refusal Removal on Decision Disposition Across Model Families [51.56484100374058]
  Abliteration - deleting a model's refusal direction from its weights - is the standard recipe behind popular "uncensored" open-weight models.<n>As a disposition probe we use 21,600 decisions under uncertainty, replayed through a frozen pipeline so the decision-layer model is the only variable.<n>Three effects replicate across both families (weeks-clustered CIs excluding zero)<n>A fourth effect reverses sign: the same operation makes Gemma-abliterated less confident and Qwen-abliterated more (family CIs non-overlapping)
  arXiv  Detail & Related papers  (2026-07-19T22:27:00Z)
• Can Code Specify a System Precisely Enough to Formally Verify It? [0.0]
  This paper evaluates both on production software: the payment workflow of an operational restaurant point-of-sale system.<n>The core protocol is correct relative to a hand-built, line-cited model under a precisely stated failure model.<n>A single probe of the production payment sandbox exposed a response-shape divergence that makes an entire recovery ladder unreachable.
  arXiv  Detail & Related papers  (2026-07-06T13:39:00Z)
• GitOfThoughts: Version-Controlled Reasoning and Agent Memory You Can Replay, Diff, and Merge [0.02066679446411895]
  We introduce GitOfThoughts, which stores an agent's reasoning tree as a git repository.<n>Every scored thought is a commit, scores are notes, outcomes are tags, and retrieval is "git log" over the agent's own history.<n>This makes reasoning replayable, auditable, and mergeable across agents at near-zero engineering cost.
  arXiv  Detail & Related papers  (2026-06-12T14:02:37Z)
• CIAware-Bench: Benchmarking Control Intervention Awareness Across Frontier LLMs [100.38986535324284]
  We introduce textbfCIAware-Bench, a benchmark for measuring textbfcontrol textbfintervention (CI) awareness across frontier models.<n>CIAware-Bench tests whether models can distinguish their own trajectories from those modified by a control intervention.
  arXiv  Detail & Related papers  (2026-06-09T16:24:16Z)
• Pramana: A Protocol-Layer Treatment of Claim Verification in Autonomous Agent Networks [0.0]
  Production verification today splits into two halves: Probabilistic verdict patterns (self-consistency voting, reviewer LLM ensembles) produce judgments, not artifacts.<n>Pramana defines the missing wire format. Every consequential agent output is wrapped in a typed ClaimAttestation with one of four variants.<n>Pramana exhaustively verified under TLC across three symmetry-reduced models: 38,563 distinct reachable states, zero invariant violations.
  arXiv  Detail & Related papers  (2026-05-19T17:00:33Z)
• Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation [87.47155146067962]
  We provide a standardized evaluation harness that orchestrates parallel evaluations across hundreds of tasks.<n>We conduct three-dimensional analysis spanning models, scaffolds, and benchmarks.<n>Our analysis reveals surprising insights, such as higher reasoning effort reducing accuracy in the majority of runs.
  arXiv  Detail & Related papers  (2025-10-13T22:22:28Z)
• Fact Checking Beyond Training Set [64.88575826304024]
  We show that the retriever-reader suffers from performance deterioration when it is trained on labeled data from one domain and used in another domain. We propose an adversarial algorithm to make the retriever component robust against distribution shift. We then construct eight fact checking scenarios from these datasets, and compare our model to a set of strong baseline models.
  arXiv  Detail & Related papers  (2024-03-27T15:15:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.