Verifiable Benchmarking of Long-Horizon Spatial Biology

• URL: http://arxiv.org/abs/2605.28065v1
• Date: Wed, 27 May 2026 07:19:18 GMT
• Title: Verifiable Benchmarking of Long-Horizon Spatial Biology
• Abstract summary: SpatialBench-Long is a benchmark for long-horizon spatial biology.<n>It tests whether AI agents can move beyond executing procedural analysis to deriving accurate scientific conclusions from complex spatial measurements.<n>Three model-harness pairs tie at 8/72 runs (11.1%): Gemini 3.5 Flash / Pi terminal coding harness, GPT-5.5 / Pi, and GPT-5.5 / OpenAI Codex.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI agents are increasingly useful for biological data analysis, but existing benchmarks mostly test broad biological knowledge, executable workflows, or localized analysis steps rather than end-to-end scientific reasoning over spatial measurements. We introduce SpatialBench-Long, a benchmark for long-horizon spatial biology in which agents must recover biological claims from raw or near-raw data and calibrated experimental context without prescribed methods. SpatialBench-Long contains 24 evaluations across primary pancreatic ductal adenocarcinoma (PDAC), engineered glioblastoma organoids and in vivo tumors, Cas9 lineage-traced lung adenocarcinoma, and mouse optic nerve aging/intervention systems, spanning CosMx, Visium, Xenium, multiplexed error-robust fluorescence in situ hybridization (MERFISH), single-cell RNA sequencing (scRNA-seq), Slide-seq, Slide-tags, histology, and lineage-recording data. Candidate claims are hardened through reproduction, independent scientist review, and trajectory inspection. Final answers are graded deterministically over controlled vocabularies and symbols with companion rubrics capturing progress through key analysis chokepoints. Across the SpatialBench-Long benchmark, three model-harness pairs tie at 8/72 runs (11.1\%): Gemini 3.5 Flash / Pi terminal coding harness, GPT-5.5 / Pi, and GPT-5.5 / OpenAI Codex. SpatialBench-Long tests whether agents can move beyond executing procedural analysis to deriving accurate scientific conclusions from complex spatial measurements.

Related papers

• When Does Deep Representation Learning Help Single-Cell Clustering? A Sensitivity-Aware Diagnostic Benchmark for Biomedical AI Pipelines [0.0]
  Single-cell ribonucleic acid sequencing (scRNA-seq) is a foundational technology for precision-medicine.<n>Is an additional deep representation stage worth its compute and tuning cost?<n>We address this question with a diagnostic benchmark of nine clustering pipelines on ten real datasets.
  arXiv  Detail & Related papers  (2026-07-28T04:48:47Z)
• Benchmarking Machine Learning Models for Multi-Omics-Based Breast Cancer Prediction [0.0]
  Estrogen Receptor (ER) status is a critical biomarker in breast cancer diagnosis, prognosis, and treatment selection.<n>This study presents a systematic benchmarking analysis of classical machine learning models for ER status prediction.
  arXiv  Detail & Related papers  (2026-06-27T02:46:40Z)
• scBench-Long: Verifiable Benchmarking of Long-Horizon Single-Cell Biology [6.518767416778027]
  We introduce scBench-Long, a benchmark for long-horizon single-cell biology.<n> scBench-Long evaluates whether agents can move beyond local analysis steps and make complex scientific claims.<n>Across 1,068 completed tasks, the strongest model--harness pair passes 16/63 runs (25.4%)
  arXiv  Detail & Related papers  (2026-06-25T03:21:50Z)
• Influence-Guided Symbolic Regression: Scientific Discovery via LLM-Driven Equation Search with Granular Feedback [56.69850045068714]
  We introduce textitInfluence-Guided Regression (IGSR), a method that frames equation discovery as an iterative two-step process.<n>We demonstrate IGSRs effectiveness on a diverse suite of benchmarks, including LLM-SRBench, pharmacological PKPD models, an epidemiological simulation, and real-world genomic data.
  arXiv  Detail & Related papers  (2026-05-27T23:48:01Z)
• scBench: Evaluating AI Agents on Single-Cell RNA-seq Analysis [6.518767416778027]
  scBench is a benchmark of 394 verifiable problems derived from scRNA-seq datasets.<n> Benchmark data on eight frontier models shows that accuracy ranges from 29-53%, with strong model-task and model-platform interactions.
  arXiv  Detail & Related papers  (2026-02-09T03:20:31Z)
• Organ-Agents: Virtual Human Physiology Simulator via LLMs [66.40796430669158]
  Organ-Agents is a multi-agent framework that simulates human physiology via LLM-driven agents.<n>We curated data from 7,134 sepsis patients and 7,895 controls, generating high-resolution trajectories across 9 systems and 125 variables.<n>Organ-Agents achieved high simulation accuracy on 4,509 held-out patients, with per-system MSEs 0.16 and robustness across SOFA-based severity strata.
  arXiv  Detail & Related papers  (2025-08-20T01:58:45Z)
• Advances in Automated Fetal Brain MRI Segmentation and Biometry: Insights from the FeTA 2024 Challenge [27.07002392996198]
  The FeTA Challenge 2024 advanced automated fetal brain MRI analysis.<n>It introduced biometry prediction as a new task alongside tissue segmentation.<n>For the first time, our diverse multi-centric test set included data from a new low-field (0.55T) MRI dataset.
  arXiv  Detail & Related papers  (2025-05-05T16:54:04Z)
• GENERator: A Long-Context Generative Genomic Foundation Model [66.46537421135996]
  We present GENERator, a generative genomic foundation model featuring a context length of 98k base pairs (bp) and 1.2B parameters.<n>Trained on an expansive dataset comprising 386B bp of DNA, the GENERator demonstrates state-of-the-art performance across both established and newly proposed benchmarks.<n>It also shows significant promise in sequence optimization, particularly through the prompt-responsive generation of enhancer sequences with specific activity profiles.
  arXiv  Detail & Related papers  (2025-02-11T05:39:49Z)
• Detecting and clustering swallow events in esophageal long-term high-resolution manometry [48.688209040613216]
  We propose a Deep Learning based swallowing detection method to accurately identify swallowing events and secondary non-deglutitive-induced esophageal motility disorders. We evaluate our computational pipeline on a total of 25 LTHRMs, which were meticulously annotated by medical experts.
  arXiv  Detail & Related papers  (2024-05-02T09:41:31Z)
• Intelligent diagnostic scheme for lung cancer screening with Raman spectra data by tensor network machine learning [10.813777115744362]
  We propose a tensor-network (TN)-ML method to reliably predict lung cancer patients and their stages via screening Raman spectra data of Volatile organic compounds (VOCs) in exhaled breath. The accuracy of the samples with high certainty is almost 100$%$.
  arXiv  Detail & Related papers  (2023-03-11T07:57:37Z)
• Generalizing electrocardiogram delineation: training convolutional neural networks with synthetic data augmentation [63.51064808536065]
  Existing databases for ECG delineation are small, being insufficient in size and in the array of pathological conditions they represent. This article delves has two main contributions. First, a pseudo-synthetic data generation algorithm was developed, based in probabilistically composing ECG traces given "pools" of fundamental segments, as cropped from the original databases, and a set of rules for their arrangement into coherent synthetic traces. Second, two novel segmentation-based loss functions have been developed, which attempt at enforcing the prediction of an exact number of independent structures and at producing closer segmentation boundaries by focusing on a reduced number of samples.
  arXiv  Detail & Related papers  (2021-11-25T10:11:41Z)
• Video-based Remote Physiological Measurement via Cross-verified Feature Disentangling [121.50704279659253]
  We propose a cross-verified feature disentangling strategy to disentangle the physiological features with non-physiological representations. We then use the distilled physiological features for robust multi-task physiological measurements. The disentangled features are finally used for the joint prediction of multiple physiological signals like average HR values and r signals.
  arXiv  Detail & Related papers  (2020-07-16T09:39:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.