Notes2Skills: From Lab Notebooks to Certainty-Aware Scientific Agent Skills

• URL: http://arxiv.org/abs/2606.11897v1
• Date: Wed, 10 Jun 2026 10:25:47 GMT
• Title: Notes2Skills: From Lab Notebooks to Certainty-Aware Scientific Agent Skills
• Abstract summary: Notes2Skills is a framework for turning lab notebooks into verifiable skills for scientific AI agents.<n>We show that Notes2Skills is the only configuration that neither mistakes uncertain notes for firm instructions nor discards firm ones.
• Score: 27.745107239660683
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Scientific discovery workflows usually contain and rely heavily on lab notes, where researchers record observations, interpret uncertain results, and plan follow-up experiments. Such informative lab notes preserve evolving scientific reasoning and author uncertainty, rather than polished final results exhibited in publications, providing a valuable opportunity for AI to engage in scientific exploration at a more comprehensive and deeper level. However, most prior work on scientific text focuses on papers, protocols, or structured databases, leaving informal laboratory notes underexplored as inputs to AI agents for science. This gap matters because lab notes often intermingle validated observations, tentative judgments, and possible experimental next steps within the same passage. If these signals are conflated, an AI agent may mistake uncertain scientific judgments for confirmed conclusions or executable actions. To this end, we present Notes2Skills, a two-stage framework for turning lab notebooks into verifiable skills for scientific AI agents while preserving the author's certainty. Across seven conditions and three wet-lab sessions, Notes2Skills is the only configuration that neither mistakes uncertain notes for firm instructions nor discards firm ones. We show that certainty preservation is the missing piece between lab notebooks and reliable agent skills, opening a path toward safer AI co-scientist systems.

Related papers

• LabAgent: Customize Any Research Hubs for Scientific Discoveries Using AI Agents [54.20891951945591]
  Methods developed by predecessors are often expanded upon by new researchers to explore more novel and in-depth scientific questions.<n>The change of lab staff, such as student graduation, leads to a lack of personnel capable of replicating methods.<n>We propose LabAgent, a reproduce and discovery harness tailored for a lab's continuous work.
  arXiv  Detail & Related papers  (2026-09-11T18:53:18Z)
• Science sandboxes measure the scientific capability of AI agents [1.1846810006828092]
  We introduce science sandboxes, a framework for studying this capability in AI agents.<n>We instantiate this framework in two biological settings, models of regulatory genomics and protein fitness prediction.<n>In particular, their scientific reasoning deteriorated when they encountered systems whose rules fell outside familiar biological priors.
  arXiv  Detail & Related papers  (2026-08-31T02:33:38Z)
• SWE-bench Science: Can Coding Agents Resolve Engineering Tasks in Science? [60.11366724565423]
  SWE-bench Science is a repository-level benchmark for scientific software engineering comprising 119 tasks from 98 GitHub repositories across 20 scientific domains.<n>We identify four recurring failure mechanisms: deficits in scientific knowledge or abstraction, misguided exploration or surface-level repair, incomplete repair coverage or system integration, and failures to generalize scientific knowledge beyond observed cases in our analysis.
  arXiv  Detail & Related papers  (2026-08-20T08:53:15Z)
• Toward Auditable AI Scientists: A Hypothesis Evolution Protocol for LLM Agents [0.42970700836450476]
  Large language model (LLM) agents are increasingly expected to play a central role in AI-driven scientific discovery.<n>Here we propose the Hypothesis Evolution Protocol (HEP), an agent harness that provides hypothesis generation, evaluation, and evolution as explicit, auditable operations.<n>On materials-science research tasks, a HEP-equipped agent operates the hypothesis--test--evidence--belief cycle that planning-style agents lack, generalizes across research questions, and exploits the protocol more fully as the base LLM becomes more capable.
  arXiv  Detail & Related papers  (2026-07-10T08:39:30Z)
• Benchmarking AI Agents for Addressing Scientific Challenges Across Scales [118.2204632627895]
  SciAgentArena is a systematic benchmark for evaluating AI agents in real-world scientific research scenarios.<n>It comprises approximately 200 tasks with stepwise verification and an interactive, agent-agnostic environment for assessing diverse AI agents.<n>We find that current agents can contribute effectively to well-specified data-analysis, particularly when the task structure and evaluation criteria are clear.
  arXiv  Detail & Related papers  (2026-06-10T22:55:30Z)
• Forecasting Scientific Progress with Artificial Intelligence [75.02347691513005]
  We evaluate scientific forecasting in AI systems through feasibility assessment, mechanistic reasoning, generative solution design, and temporal prediction.<n>Across 4,760 scientific events, we observe systematic and domain-dependent limitations in current frontier models.<n>Performance is highly heterogeneous across domains, with the timing of AI progress more predictable than advances in biology, chemistry, and physics.
  arXiv  Detail & Related papers  (2026-05-21T16:23:36Z)
• Federated Semantic Knowledge Graphs for Laboratory Workflows: A Structured Expert Elicitation Methodology Demonstrated Through Bioanalytical Workflow Twins [0.0]
  Laboratory in pharmaceutical and biomedical research encode substantial tacit knowledge.<n>We present a structured expert elicitation methodology and federated Semantic Knowledge Graph (SKG) architecture for capturing and querying this knowledge.<n>This architecture provides the semantic world model that AI laboratory agents lack a queryable representation of where human judgment is irreplaceable, and which execution assets mask rather than detect failure.
  arXiv  Detail & Related papers  (2026-05-15T18:44:52Z)
• Experiment-as-Code Labs: A Declarative Stack for AI-Driven Scientific Discovery [16.75156456478123]
  We propose a new paradigm called Experiment-as-Code (EaC) Labs''<n>EaC Labs encode experiments as declarative configurations that can be compiled down to device-level APIs.<n>This is a general stack that is science-, lab-, and instrument-independent.
  arXiv  Detail & Related papers  (2026-05-06T00:50:34Z)
• When AI reviews science: Can we trust the referee? [73.47745294608072]
  We map attacks across the review lifecycle -- training and data retrieval, desk review, deep review, rebuttal, and system-level.<n>We use two advanced LLM-based referees to isolate the causal effects of prestige framing, assertion strength, rebuttal sycophancy, and contextual poisoning on review scores.
  arXiv  Detail & Related papers  (2026-04-26T08:03:32Z)
• Embodied Science: Closing the Discovery Loop with Agentic Embodied AI [73.54118756665221]
  We argue for embodied science, a paradigm that reframes scientific discovery as a closed loop tightly coupling agentic reasoning with physical execution.<n>We propose a unified Perception-Language-Action-Discovery (PLAD) framework, wherein embodied agents perceive experimental environments, reason over scientific knowledge, execute physical interventions, and internalize outcomes to drive subsequent exploration.
  arXiv  Detail & Related papers  (2026-03-20T09:20:12Z)
• Bohrium + SciMaster: Building the Infrastructure and Ecosystem for Agentic Science at Scale [82.20980951765891]
  We argue that scaling agentic science requires an infrastructure-and-ecosystem approach, instantiated Bohrium+SciMaster.<n>Bohrium acts as a managed, traceable hub for AI4S assets that turns diverse scientific data, software, compute, and laboratory systems into agent-ready capabilities.<n>SciMaster orchestrates these capabilities into long-horizon scientific, on which scientific agents can be composed and executed.
  arXiv  Detail & Related papers  (2025-12-23T16:04:41Z)
• AstaBench: Rigorous Benchmarking of AI Agents with a Scientific Research Suite [75.58737079136942]
  We present AstaBench, a suite that provides the first holistic measure of agentic ability to perform scientific research.<n>Our suite comes with the first scientific research environment with production-grade search tools.<n>Our evaluation of 57 agents across 22 agent classes reveals several interesting findings.
  arXiv  Detail & Related papers  (2025-10-24T17:10:26Z)
• The More You Automate, the Less You See: Hidden Pitfalls of AI Scientist Systems [11.543423308064275]
  AI scientist systems are capable of executing the full research workflow from hypothesis generation to paper writing.<n>This lack of scrutiny poses a risk of introducing flaws that could undermine the integrity, reliability, and trustworthiness of their research outputs.<n>We identify four potential failure modes in contemporary AI scientist systems: inappropriate benchmark selection, data leakage, metric misuse, and post-hoc selection bias.
  arXiv  Detail & Related papers  (2025-09-10T16:04:24Z)
• Autonomous Microscopy Experiments through Large Language Model Agents [4.241267255764773]
  Large language models (LLMs) are revolutionizing self driving laboratories (SDLs) for materials research.<n>We introduce Artificially Intelligent Lab Assistant (AILA), a framework automating atomic force microscopy through LLM driven agents.<n>We find that state of the art models struggle with basic tasks and coordination scenarios.
  arXiv  Detail & Related papers  (2024-12-18T09:35:28Z)
• DISCOVERYWORLD: A Virtual Environment for Developing and Evaluating Automated Scientific Discovery Agents [49.74065769505137]
  We introduce DISCOVERYWORLD, the first virtual environment for developing and benchmarking an agent's ability to perform complete cycles of novel scientific discovery. It includes 120 different challenge tasks spanning eight topics each with three levels of difficulty and several parametric variations. We find that strong baseline agents, that perform well in prior published environments, struggle on most DISCOVERYWORLD tasks.
  arXiv  Detail & Related papers  (2024-06-10T20:08:44Z)
• "Turing Tests" For An AI Scientist [0.0]
  This paper proposes a "Turing test for an AI scientist" to assess whether an AI agent can conduct scientific research independently. We propose seven benchmark tests that evaluate an AI agent's ability to make groundbreaking discoveries in various scientific domains.
  arXiv  Detail & Related papers  (2024-05-22T05:14:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.