Artifact-centered Claim-aware Observability for Autonomous Scientific Agents

• URL: http://arxiv.org/abs/2608.18312v1
• Date: Tue, 18 Aug 2026 20:47:24 GMT
• Title: Artifact-centered Claim-aware Observability for Autonomous Scientific Agents
• Abstract summary: We propose a compact observability profile organized around individuals, operators, fitness records, lineage, archives, runs, streams, and steering commands.<n>The profile is intended as a semantic layer that complements current telemetry and provenance standards.
• Score: 4.403170275941483
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Autonomous scientific agents now increasingly propose ideas, write code, run experiments, analyze results, and even draft papers. Observe and audit those agents are necessary but logging every model call is not enough, scientists also need to inspect the artifacts and claims that the systems produced and their relations. This is driven by the fact that failures in scientific agent systems are often distributed across several objects. A manuscript claim may cite the wrong evidence, a search process may select a degenerate candidate, a laboratory novelty claim may depend on an unstated rule, or a multi-agent plan may change without a visible trigger. Existing tracing, experiment tracking, and archival provenance tools are valuable, but their native objects do not make these scientific audit relations first-class. We argue that autonomous scientific systems should emit portable, claim-aware artifact lineage as a minimum audit layer. We propose a compact observability profile organized around individuals, operators, fitness records, lineage, archives, runs, streams, and steering commands. In this profile, scientific claims are ordinary individuals with explicit evidence bindings and verification records. The profile is intended as a semantic layer that complements current telemetry and provenance standards. Execution details can remain in OpenTelemetry. Final packages can export to PROV-O or RO-Crate standards.

Related papers

• EarthVerse: Benchmarking Scientific Agents Across Dynamic Earth Systems and Natural Hazards [73.12811119273206]
  We introduce EarthVerse, a benchmark that evaluates scientific agents through package-scoped investigations.<n>Agents inspect heterogeneous event packages, choose compatible evidence, execute transparent calculations, reconcile source differences, and preserve provenance in the final answer.<n>Across systems, the best mean answer-unit accuracy is 84.65%, while the highest Strict@95 is only 34.81%.
  arXiv  Detail & Related papers  (2026-08-24T17:29:16Z)
• SciForge: An AI-Native, Multimodal Workbench for Scientific Discovery [48.99744520105086]
  SciForge is a multimodal research-native AI workbench.<n>It reserves the graphical interface for human judgment while search, parsing, model routing, workflow execution, plotting, writing, and presentation generation run as modular agent-accessible services.
  arXiv  Detail & Related papers  (2026-07-17T15:13:03Z)
• XScientist: A Git-Like Research Protocol for Long-Running Autonomous Scientific Discovery [1.1923076571060318]
  XScientist is a git-like research protocol and operating system for autonomous research systems.<n>It orchestrates idea generation, experiment execution, manuscript drafting, self-review, repair, quality gating, daemon scheduling, and artifacts as one continuously observable pipeline.<n>It exports an Agent-Native Research Artifact protocol that records an exploration DAG, per-node code and outputs, claim-to-evidence anchors, content hashes, provenance, and re-execution hooks.
  arXiv  Detail & Related papers  (2026-07-14T03:20:11Z)
• Rethinking Scientific Discovery in the Agentic Era [69.85050591628048]
  SCION is an agentic scientific operating system that acts as an textbf organizational nexus.<n>At its core is the textbfResearch Execution Plan (REP), which compiles high-level scientific intent into staged objectives, dependencies, verification checkpoints, tool requirements, expected artifacts, and fallback conditions.<n>We formulate discovery under SCION as textbfTarget-conditioned Inverse Search and extend it to hidden-target settings through batch active search under finite experimental budgets.
  arXiv  Detail & Related papers  (2026-07-04T13:09:33Z)
• Position: Correct Answer, Wrong Mechanism -- When AI Scientists Defend General Claims Their Own Data Contradicts [0.0]
  Coding agents prove reliable tools but unreliable scientific co-authors for open-ended claim-making.<n>A one-step regime-shift check needs only the agent's claim and flags the over-generalized cases.<n>A companion recomputation flags the remaining cases when the correct observable is known.
  arXiv  Detail & Related papers  (2026-06-22T11:14:07Z)
• The Age of AI Agents Demands A New Scientific Paradigm To Sustain Trustworthy Science [0.0]
  We argue that science must evolve its verification infrastructure, as it has before with peer review.<n>We propose criteria for an adapted verification infrastructure that emphasizes observable-by-default verification, scalable results, and clear attribution.
  arXiv  Detail & Related papers  (2026-06-18T20:38:50Z)
• Externalizing Research Synthesis and Validation in AI Scientists through a Research Harness [22.631467433443362]
  Xcientist is a research harness that externalizes research synthesis and experimental validation into inspectable, contract-governed processes.<n>Xcientist literature organizes evidence, idea states, implementation plans, ablation records and repair traces as persistent research artifacts.
  arXiv  Detail & Related papers  (2026-06-17T09:52:14Z)
• The Last Human-Written Paper: Agent-Native Research Artifacts [106.47848184955576]
  We introduce the Agent-Native Research Artifact (ARA), a protocol that replaces the narrative paper with a machine-executable research package.<n>Three mechanisms support the ecosystem: a Live Research Manager that captures decisions and dead ends during ordinary development; an ARA Compiler that translates legacy PDFs and repos into ARAs; and an ARA-native review system that automates objective checks so human reviewers can focus on significance, novelty, and taste.
  arXiv  Detail & Related papers  (2026-04-27T16:23:09Z)
• CiteAudit: You Cited It, But Did You Read It? A Benchmark for Verifying Scientific References in the LLM Era [51.63024682584688]
  Large language models (LLMs) introduce a new risk: fabricated references that appear plausible but correspond to no real publications.<n>We present the first comprehensive benchmark and detection framework for hallucinated citations in scientific writing.<n>Our framework significantly outperforms prior methods in both accuracy and interpretability.
  arXiv  Detail & Related papers  (2026-02-26T19:17:39Z)
• Bohrium + SciMaster: Building the Infrastructure and Ecosystem for Agentic Science at Scale [82.20980951765891]
  We argue that scaling agentic science requires an infrastructure-and-ecosystem approach, instantiated Bohrium+SciMaster.<n>Bohrium acts as a managed, traceable hub for AI4S assets that turns diverse scientific data, software, compute, and laboratory systems into agent-ready capabilities.<n>SciMaster orchestrates these capabilities into long-horizon scientific, on which scientific agents can be composed and executed.
  arXiv  Detail & Related papers  (2025-12-23T16:04:41Z)
• RerrFact: Reduced Evidence Retrieval Representations for Scientific Claim Verification [4.052777228128475]
  We propose a modular approach that sequentially carries out binary classification for every prediction subtask. We carry out two-step stance predictions that first differentiate non-relevant rationales and then identify supporting or refuting rationales for a given claim. Experimentally, our system RerrFact with no fine-tuning, simple design, and a fraction of model parameters fairs competitively on the leaderboard.
  arXiv  Detail & Related papers  (2022-02-05T21:52:45Z)
• Fact or Fiction: Verifying Scientific Claims [53.29101835904273]
  We introduce scientific claim verification, a new task to select abstracts from the research literature containing evidence that SUPPORTS or REFUTES a given scientific claim. We construct SciFact, a dataset of 1.4K expert-written scientific claims paired with evidence-containing abstracts annotated with labels and rationales. We show that our system is able to verify claims related to COVID-19 by identifying evidence from the CORD-19 corpus.
  arXiv  Detail & Related papers  (2020-04-30T17:22:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.