Evaluating Agentic Bioinformatics through Function, Evidence, and Validation

• URL: http://arxiv.org/abs/2607.27556v1
• Date: Thu, 30 Jul 2026 01:00:36 GMT
• Title: Evaluating Agentic Bioinformatics through Function, Evidence, and Validation
• Abstract summary: Large language model agents increasingly plan, execute, and interpret biological analyses, yet fluent responses, successful tool calls, and benchmark performance alone do not establish scientific credibility.<n>We introduce the Function-Evidence-Validation (FEV) framework, which separates demonstrated workflow operations, traceable support for actions and claims, and use-case-specific validation.<n>Using FEV, we map 109 agentic or agent-adjacent systems and 28 benchmark or evaluation resources, representing 128 unique publications across genomics, single-cell and spatial omics, protein science, drug discovery, computational pathology, and general bioinformatics automation.
• Score: 6.840730950777982
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language model agents increasingly plan, execute, and interpret biological analyses, yet fluent responses, successful tool calls, and benchmark performance alone do not establish scientific credibility. Existing reviews primarily organize biological agents by application, architecture, and agentic capability, but do not jointly operationalize the accountability of agent-generated workflows. We address this gap by treating the inspectable workflow trajectory, rather than architecture or final output alone, as the primary unit of analysis. We introduce the Function--Evidence--Validation (FEV) framework, which separates demonstrated workflow operations, traceable support for actions and claims, and use-case-specific validation. Using FEV, we map 109 agentic or agent-adjacent systems and 28 benchmark or evaluation resources, representing 128 unique publications across genomics, single-cell and spatial omics, protein science, drug discovery, computational pathology, and general bioinformatics automation. Across domains, planning and tool-mediated execution have advanced more rapidly than replayability, provenance, robust scientific assessment, external validation, and prospective empirical testing. We therefore argue that agentic bioinformatics should be assessed through workflow correctness rather than final-answer correctness alone. FEV provides a practical basis for comparing systems and designing transparent, auditable, and scientifically accountable bioinformatics workflows.

Related papers

• Agentic AI in medicine: architectures, applications, evaluation, and challenges for clinical translation [16.678400738326236]
  The scope of agentic AI in medicine remains unsettled, and current evaluation practices are not yet aligned with the requirements of clinical use.<n>We conducted a scoping review with systematic evidence mapping across five electronic sources, screened 1,649 exportable records, and provisionally included 557 unique studies.<n>The included studies describe single agents that use external tools, supported by retrieval and external knowledge, multimodal agents, and multi-agent systems applied to medical question answering, image interpretation, electronic health record analysis, drug safety, and clinical trial prediction.
  arXiv  Detail & Related papers  (2026-07-28T09:25:25Z)
• AutoResearch AI: Towards AI-Powered Research Automation for Scientific Discovery [184.2309450554096]
  AutoResearch is defined as the developmental spectrum of AI-powered scientific workflow automation.<n>We analyze how research systems redistribute control, evidence, execution, validation, and accountability across and organize the field.<n>We propose five evaluation dimensions--novelty, impact, reliability, and provenance--and show that AutoResearch autonomy is domain-conditioned.
  arXiv  Detail & Related papers  (2026-05-22T03:40:30Z)
• Towards a Virtual Neuroscientist: Autonomous Neuroimaging Analysis via Multi-Agent Collaboration [53.772014300855375]
  We introduce NIAgent, a multi-agent system for autonomous end-to-end neuroimaging analysis.<n>Unlike conventional flat tool-calling agents, NIAgent adopts a code-centric execution paradigm.<n>We propose a hierarchical verification framework for autonomous quality control, integrating cohort-level metric screening with agentic visual inspection.
  arXiv  Detail & Related papers  (2026-05-10T06:30:19Z)
• Can Coding Agents Reproduce Findings in Computational Materials Science? [49.254975563645786]
  We present AutoMat, a benchmark for evaluating large language models' ability to reproduce scientific claims.<n>By working closely with subject matter experts, we curate a set of claims from real materials science papers to test whether coding agents can recover and execute the end-to-end workflow.<n>Results show that current LLM-based agents obtain low overall success rates on AutoMat, with the best-performing setting achieving a success rate of only 54.1%.
  arXiv  Detail & Related papers  (2026-05-01T17:42:12Z)
• Towards Agentic Intelligence for Materials Science [73.4576385477731]
  This survey advances a unique pipeline-centric view that spans from corpus curation and pretraining to goal-conditioned agents interfacing with simulation and experimental platforms.<n>To bridge communities and establish a shared frame of reference, we first present an integrated lens that aligns terminology, evaluation, and workflow stages across AI and materials science.
  arXiv  Detail & Related papers  (2026-01-29T23:48:43Z)
• BiomechAgent: AI-Assisted Biomechanical Analysis Through Code-Generating Agents [1.1458853556386797]
  We present BiomechAgent, a code-generating AI agent that enables biomechanical analysis through natural language.<n>We developed a benchmark spanning data retrieval, visualization, activity classification, temporal segmentation, and clinical reasoning.<n>Biomechanically-informed, domain-specific instructions significantly improved performance over generic prompts.
  arXiv  Detail & Related papers  (2026-01-16T04:30:04Z)
• SpatialBench: Can Agents Analyze Real-World Spatial Biology Data? [6.993633248897315]
  We introduce SpatialBench, a benchmark of 146 verifiable problems derived from practical spatial analysis.<n>Each problem provides a snapshot of experimental data immediately prior to an analysis step.<n>Base model accuracy remains low, with strong model-task and model-platform interactions.
  arXiv  Detail & Related papers  (2025-12-26T07:40:11Z)
• An Agentic Framework for Autonomous Materials Computation [70.24472585135929]
  Large Language Models (LLMs) have emerged as powerful tools for accelerating scientific discovery.<n>Recent advances integrate LLMs into agentic frameworks, enabling retrieval, reasoning, and tool use for complex scientific experiments.<n>Here, we present a domain-specialized agent designed for reliable automation of first-principles materials computations.
  arXiv  Detail & Related papers  (2025-12-22T15:03:57Z)
• Simple Agents Outperform Experts in Biomedical Imaging Workflow Optimization [69.36509281190662]
  Adapting production-level computer vision tools to bespoke scientific datasets is a critical "last mile" bottleneck.<n>We consider using AI agents to automate this manual coding, and focus on the open question of optimal agent design.<n>We demonstrate that a simple agent framework consistently generates adaptation code that outperforms human-expert solutions.
  arXiv  Detail & Related papers  (2025-12-02T18:42:26Z)
• Biomedical reasoning in action: Multi-agent System for Auditable Biomedical Evidence Synthesis [19.14228623181563]
  We present M-Reason, a demonstration system for transparent, agent-based reasoning and evidence integration in the biomedical domain.<n>M-Reason leverages recent advances in large language models (LLMs) and modular agent orchestration to automate evidence retrieval, appraisal, and synthesis across diverse biomedical data sources.<n>An open, interactive user interface allows researchers to directly observe, explore and evaluate the multi-agent workflow.
  arXiv  Detail & Related papers  (2025-10-06T19:57:29Z)
• LLM Agent Swarm for Hypothesis-Driven Drug Discovery [2.7036595757881323]
  PharmaSwarm is a unified multi-agent framework that orchestrates specialized "agents" to propose, validate, and refine hypotheses for novel drug targets and lead compounds.<n>By acting as an AI copilot, PharmaSwarm can accelerate translational research and deliver high-confidence hypotheses more efficiently than traditional pipelines.
  arXiv  Detail & Related papers  (2025-04-24T22:27:50Z)
• BioDiscoveryAgent: An AI Agent for Designing Genetic Perturbation Experiments [112.25067497985447]
  We introduce BioDiscoveryAgent, an agent that designs new experiments, reasons about their outcomes, and efficiently navigates the hypothesis space to reach desired solutions.<n>BioDiscoveryAgent can uniquely design new experiments without the need to train a machine learning model.<n>It achieves an average of 21% improvement in predicting relevant genetic perturbations across six datasets.
  arXiv  Detail & Related papers  (2024-05-27T19:57:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.