From Trajectories to Evidence: Auditable Experimental Records for Industrial Research Agents

• URL: http://arxiv.org/abs/2608.05235v1
• Date: Wed, 05 Aug 2026 13:37:38 GMT
• Title: From Trajectories to Evidence: Auditable Experimental Records for Industrial Research Agents
• Abstract summary: Research agents increasingly conduct multi-round machine-learning experiments in industrial recommendation settings.<n>Yet a completed trajectory is not automatically evidence: generated artifacts may be unsupported or incomplete, executed rounds may be invalid or confounded, and later modifications may obscure earlier findings.<n>We introduce an evidence-grounded framework that couples bounded verification of consequential artifacts with post-execution claim qualification.
• Score: 47.678562263387214
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Research agents increasingly conduct multi-round machine-learning experiments in industrial recommendation settings and retain the resulting trajectories to guide later decisions. Yet a completed trajectory is not automatically evidence: generated artifacts may be unsupported or incomplete, executed rounds may be invalid or confounded, and later modifications may obscure earlier findings. We study \textbf{trajectory-to-evidence conversion}, asking what a completed research process has actually established. We introduce an evidence-grounded framework that couples bounded verification of consequential artifacts with post-execution claim qualification. A context-isolated generate--verify--repair process checks artifacts for evidence violations and missing downstream requirements before release. After execution, validity and attribution checks consolidate evidence across rounds, qualify intervention-level claims as actionable repairs, diagnostic guards, or withheld findings, and preserve admitted claims as auditable records with explicit provenance and applicability boundaries. A hybrid LLM-assisted controller subsequently applies, defers, or rejects records based on available target evidence. Record audits characterize which claims survive qualification, while downstream diagnostics identify affirmative applicability judgment as a bottleneck for the tested controller. Across paper-to-target adaptations, later rounds often improve on the first, while final rounds frequently underperform an earlier best, exposing non-monotonic trajectory evolution. Candidates produced through the complete workflow also yielded positive online lifts relative to deployed baselines.

Related papers

• The Verification Gap in Networked Physical AI: A Post-Semantic Communication Framework [0.7896651877485362]
  In networked Physical AI, a proposal may be understood while valid, timely, proposal-bound evidence or the authority required to finalize an action remains unavailable.<n>We call this mismatch the verification gap and introduce a Post-Semantic Communication Framework for the systems interface between proposal formation and physical execution.
  arXiv  Detail & Related papers  (2026-08-20T03:17:20Z)
• DRNOISE: Benchmarking Deep Research Agents in Misleading Evidence Environments [51.049999642138]
  We introduce DRNOISE, a 100-task benchmark for answer recovery under misleading evidence.<n>Each task has a unique gold answer supported by two corroborating indirect record chains; the paired noisy condition adds one plausible document that states a conflicting answer directly.<n>Across agents with strong clean-task performance, this single intervention causes 66-88 percentage-point accuracy drops.
  arXiv  Detail & Related papers  (2026-07-19T15:20:40Z)
• Diagnosing LLM Arbitration Behavior over Pre-evidence Epistemic States in RAG-based Fact-Checking [14.164769475625391]
  In RAG-based fact-checking, LLMs are increasingly used as verifiers to check given claims against retrieved evidence.<n>We introduce textscPAVE (emphPrior-Aware Verifier Evaluation), a diagnostic testbed that stratifies an LLM verifier into four epistemic states.<n>Experiments reveal unreliable and highly model-dependent prior-context arbitration, highlighting the importance of verifier selection for real-world RAG-based fact-checking applications.
  arXiv  Detail & Related papers  (2026-05-31T09:37:56Z)
• PAVE: Premise-Aware Validation and Editing for Retrieval-Augmented LLMs [8.082352336629816]
  PAVE is an inference-time validation layer for evidence-grounded question answering.<n>It decomposes retrieved context into question-conditioned atomic facts, drafts an answer, scores how well that draft is supported by the extracted premises, and revises low-support outputs before finalization.
  arXiv  Detail & Related papers  (2026-03-21T06:23:21Z)
• From Fluent to Verifiable: Claim-Level Auditability for Deep Research Agents [8.49451413641847]
  We argue that as research generation becomes cheap, auditability becomes the bottleneck.<n>This perspective proposes claim-level auditability as a first-class design and evaluation target for deep research agents.
  arXiv  Detail & Related papers  (2026-02-14T19:39:15Z)
• Preventing the Collapse of Peer Review Requires Verification-First AI [49.995126139461085]
  We propose truth-coupling, i.e. how tightly venue scores track latent scientific truth.<n>We formalize two forces that drive a phase transition toward proxy-sovereign evaluation.
  arXiv  Detail & Related papers  (2026-01-23T17:17:32Z)
• Retrieve-Refine-Calibrate: A Framework for Complex Claim Fact-Checking [32.6738019397553]
  We propose a Retrieve-Refine-Calibrate (RRC) framework based on large language models (LLMs)<n>Specifically, the framework first identifies the entities mentioned in the claim and retrieves evidence relevant to them.<n>Then, it refines the retrieved evidence based on the claim to reduce irrelevant information.<n>Finally, it calibrates the verification process by re-evaluating low-confidence predictions.
  arXiv  Detail & Related papers  (2026-01-23T08:48:52Z)
• Contrastive Learning to Improve Retrieval for Real-world Fact Checking [84.57583869042791]
  We present Contrastive Fact-Checking Reranker (CFR), an improved retriever for fact-checking complex claims. We leverage the AVeriTeC dataset, which annotates subquestions for claims with human written answers from evidence documents. We find a 6% improvement in veracity classification accuracy on the dataset.
  arXiv  Detail & Related papers  (2024-10-07T00:09:50Z)
• From Relevance to Utility: Evidence Retrieval with Feedback for Fact Verification [118.03466985807331]
  We argue that, rather than relevance, for FV we need to focus on the utility that a claim verifier derives from the retrieved evidence.<n>We introduce the feedback-based evidence retriever(FER) that optimize the evidence retrieval process by incorporating feedback from the claim verifier.
  arXiv  Detail & Related papers  (2023-10-18T02:59:38Z)
• Complex Claim Verification with Evidence Retrieved in the Wild [73.19998942259073]
  We present the first fully automated pipeline to check real-world claims by retrieving raw evidence from the web. Our pipeline includes five components: claim decomposition, raw document retrieval, fine-grained evidence retrieval, claim-focused summarization, and veracity judgment.
  arXiv  Detail & Related papers  (2023-05-19T17:49:19Z)
• AmbiFC: Fact-Checking Ambiguous Claims with Evidence [57.7091560922174]
  We present AmbiFC, a fact-checking dataset with 10k claims derived from real-world information needs. We analyze disagreements arising from ambiguity when comparing claims against evidence in AmbiFC. We develop models for predicting veracity handling this ambiguity via soft labels.
  arXiv  Detail & Related papers  (2021-04-01T17:40:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.