Related papers: Plato-Bio: verification-first biological novelty screening with temporal rediscovery and structural benchmarks

Plato-Bio: verification-first biological novelty screening with temporal rediscovery and structural benchmarks

URL: http://arxiv.org/abs/2607.23975v1
Date: Mon, 27 Jul 2026 03:55:39 GMT
Title: Plato-Bio: verification-first biological novelty screening with temporal rediscovery and structural benchmarks
Abstract summary: We developed Plato-Bio, a biology-routed extension of the open Plato/Denario architecture.<n>Plato-Bio couples explicit workflow states with provenance records, citation checks, claim-to-evidence links, scoped file writes, and publication gates.<n>On the current clean revision, the full Python suite completed with 931 passes, six skips, and no failures or errors.
Score: 0.0
License: http://creativecommons.org/licenses/by/4.0/
Abstract: Large language model research agents can connect literature retrieval, analysis code, and manuscript preparation, but coherent output does not establish scientific validity. We developed Plato-Bio, a biology-routed extension of the open Plato/Denario architecture that couples explicit workflow states with provenance records, citation checks, claim-to-evidence links, scoped file writes, and publication gates. A source audit identified and repaired three defects that could distort evaluation: loss of task domain in the default factory, omission of declared method signals from scoring, and evidence sidecars that lacked the drafted-claim denominator. On the current clean revision, the full Python suite completed with 931 passes, six skips, and no failures or errors; targeted biology, genomics, evidence/citation, and adversarial-safety suites likewise completed without failure. We evaluated two narrow use cases. In a frozen historical rediscovery task, independent pre-1986 literature bridges ranked the later-studied relation between fish oil and Raynaud phenomenon first; TF-IDF ranked it second and corpus frequency third. This single curated task measures retrospective ranking, not prospective discovery. In a separate comparison of AlphaFold models with experimental structures for 15 human proteins, 11 targets had high-confidence-core C-alpha RMSD below 1 Angstrom (median 0.501 Angstrom). Four targets exceeded 2 Angstrom, and confidence masking reduced the SUMO1 discrepancy from 16.61 to 2.58 Angstrom over 74 residues. The workflow emitted 27 traceable discrepancy regions, all retained as unvalidated hypotheses. Plato-Bio therefore provides reproducible software contracts and auditable screening baselines; broader claims of agent efficacy or biological novelty require preregistered evaluation, independent review, and prospective validation.

Related papers

PaperDoctor: Evidence-Grounded and Actionable Feedback for Scientific Papers in Progress [130.58520199343707]
PaperDoctor is an agent framework for pre-submission feedback with three key innovations.<n>First, a holistic hierarchical framework evaluates writing, layout, references, code, theory, prior work, and experiments.<n>Second, each finding contains an observation, a pointer to specific evidence such as a sentence, equation, or code line, and a revision suggestion.<n>Third, PaperDoctor selectively rebuilds and reruns experiments based on claim importance and compute budget.
arXiv Detail & Related papers (2026-09-15T11:08:11Z)
NovGauge: A Fine-Grained Benchmark for Diagnosing LLMs' Capability in Paper Novelty Assessment [54.4265627247104]
Large language models (LLMs) are increasingly used in peer review at major AI conferences.<n>Existing benchmarks assess novelty as a single holistic score.<n>We present NovGauge, a human-anchored benchmark for fine-grained novelty assessment diagnosis.
arXiv Detail & Related papers (2026-09-10T08:34:56Z)
VeriPhy: Agentic Physical Reasoning for World Model Evaluation and Refinement [57.86962208273888]
We present an auditable physical-verification system in which a text-only planner compiles the prompt into typed physical obligations.<n>Each action returns a provenance-carrying evidence record whose payload, when usable, is either a typed measurement or an explicitly tagged learned state.<n>We anchor evaluation in a 1,500-clip corpus of human-annotated flaw records that localize real generation failures in prompt reference, space, and time.
arXiv Detail & Related papers (2026-09-02T20:36:45Z)
TRACES: A Benchmark for Epistemic Reliability in Scientific Reasoning by LLMs [0.0]
Large language models are being proposed as agents in scientific, in domains where no downstream verifier exists.<n>We introduce a probe corpus of 42 retracted, fraudulent, and pseudoscientific papers, paired with a methodology for eliciting and scoring single-shot model engagement with each paper's framing.<n>The probes span five claim types: fabricated observation, pseudophysical mechanism, magical premise, legitimization bridge, and cargo-cult experiment.
arXiv Detail & Related papers (2026-08-11T20:30:32Z)
Fantastic Adaptive Taxonomies and How to Use Them [100.20614173157708]
AdaMAST converts execution traces into a compact, evidence-grounded failure taxonomy.<n>No code is hand-authored, no trace human-annotated.<n>In agent-system search, taxonomy-coded diagnoses of failed candidates outperform free-form.
arXiv Detail & Related papers (2026-07-17T17:41:16Z)
Before the Action: Benchmarking LLMs on Prospective Hypothesis Discovery [95.44432473367836]
Large language models (LLMs) excel at answering pre-specified questions, yet their ability to navigate the open-ended, pre-conclusion stage of discovery remains largely unmeasured.<n>We introduce Prospective Hypothesis Discovery (PHD), which asks models to autonomously construct grounded, discnative, and testable hypothesis spaces from inconclusive evidence.<n>We introduce HypoArena, comprising HypoData, a benchmark of 988 cases across six scientific and analytical domains, and HypoEval, an evaluation framework for open-ended hypothesis sets.
arXiv Detail & Related papers (2026-07-17T08:56:43Z)
One Reflection Is Not Enough: Self-Correcting Autonomous Research via Multi-Hypothesis Failure Attribution [85.65263343588026]
We propose SAGE, a Self-correcting, Autonomous, Grounded Experimenter.<n>Its core mechanism, Multi-Hypothesis Failure Attribution (MHFA), treats recovery as a structured causal diagnosis.<n>On a 12-topic, 5-domain benchmark, SAGE increases metrics-bearing outputs from 42% to 92% over a baseline reflection.
arXiv Detail & Related papers (2026-06-30T10:54:16Z)
Deterministic Integrity Gates for LLM-Assisted Clinical Manuscript Preparation: An Auditable Biomedical Informatics Architecture [0.3823356975862005]
We describe an architecture pairing generation with verification, resting on three principles.<n>This determinism-where-possible split, organized as an integrity-gate taxonomy, is the core contribution.<n> MedSci Skills is an open-source toolkit of 43 skills with a 21-detector deterministic tier, evaluated on three public-dataset pipelines.
arXiv Detail & Related papers (2026-06-08T13:51:04Z)
AEGIS: From Clues to Verdicts -- Graph-Guided Deep Vulnerability Reasoning via Dialectics and Meta-Auditing [9.271196825503417]
Large Language Models (LLMs) are increasingly adopted for vulnerability detection, yet their reasoning remains fundamentally unsound.<n>We present AEGIS, a novel multi-agent framework that shifts detection from ungrounded speculation to forensic verification over a closed factual substrate.<n>It reduces the false positive rate by up to 54.40% compared to leading baselines, at an average cost of $0.09 per sample without any task-specific training.
arXiv Detail & Related papers (2026-03-21T04:12:04Z)
Negation is Not Semantic: Diagnosing Dense Retrieval Failure Modes for Trade-offs in Contradiction-Aware Biomedical QA [1.0330395403064265]
Large Language Models (LLMs) have demonstrated strong capabilities in question answering, yet their tendency to generate plausible but unverified claims poses serious risks in clinical settings.<n>To mitigate these risks, the TREC 2025 BioGen track mandates grounded answers that explicitly surface contradictory evidence.<n>We present a proxy-based development framework using the SciFact dataset to systematically optimize retrieval architectures.
arXiv Detail & Related papers (2026-03-18T10:35:44Z)
VulAgent: Hypothesis-Validation based Multi-Agent Vulnerability Detection [55.957275374847484]
VulAgent is a multi-agent vulnerability detection framework based on hypothesis validation.<n>It implements a semantics-sensitive, multi-view detection pipeline, each aligned to a specific analysis perspective.<n>On average, VulAgent improves overall accuracy by 6.6%, increases the correct identification rate of vulnerable--fixed code pairs by up to 450%, and reduces the false positive rate by about 36%.
arXiv Detail & Related papers (2025-09-15T02:25:38Z)
Fact or Facsimile? Evaluating the Factual Robustness of Modern Retrievers [34.31192184496381]
Dense retrievers and rerankers are central to retrieval-augmented generation (RAG) pipelines.<n>We evaluate how much factual competence these components inherit or lose from large language models (LLMs) they are based on.<n>For every embedding model, cosine-similarity scores between queries and correct completions are significantly higher than those for incorrect ones.
arXiv Detail & Related papers (2025-08-28T04:13:51Z)
Beyond Memorization: Reasoning-Driven Synthesis as a Mitigation Strategy Against Benchmark Contamination [77.69093448529455]
We present an empirical study using an infinitely scalable framework to synthesize research-level QA directly from arXiv papers.<n>We evaluate a lack of significant performance decay near knowledge cutoff dates for models of various sizes, developers, and release dates.<n>We hypothesize that the multi-step reasoning required by our synthesis pipeline offered additional complexity that goes deeper than shallow memorization.
arXiv Detail & Related papers (2025-08-26T16:41:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.