scBench-Long: Verifiable Benchmarking of Long-Horizon Single-Cell Biology

• URL: http://arxiv.org/abs/2606.26563v1
• Date: Thu, 25 Jun 2026 03:21:50 GMT
• Title: scBench-Long: Verifiable Benchmarking of Long-Horizon Single-Cell Biology
• Abstract summary: We introduce scBench-Long, a benchmark for long-horizon single-cell biology.<n> scBench-Long evaluates whether agents can move beyond local analysis steps and make complex scientific claims.<n>Across 1,068 completed tasks, the strongest model--harness pair passes 16/63 runs (25.4%)
• Score: 6.518767416778027
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Single-cell studies require analysts to convert raw measurements into specific biological claims through multi-step workflows and integration of metadata, assay context, and auxiliary evidence. Existing AI-biology benchmarks largely measure broad knowledge, executable workflows, or local analysis steps. We introduce scBench-Long, a benchmark for long-horizon single-cell biology in which agents must recover scientific conclusions from raw or near-raw data without prescribed methods. The benchmark contains 21 evaluations spanning melanoma CD8 T-cell reactivity, CD8 RNA+ATAC regulatory inference, human--monkey chimera development, KRAS-driven lung tumor aging, and lethal COVID-19 lung pathology. Tasks cover paired scRNA/TCR sequencing, RNA and chromatin profiling, cross-species transcriptomics, combinatorial scRNA-seq, single-nucleus RNA-seq, immune repertoires, ortholog maps, ligand--receptor resources, and validation evidence. Candidate claims are reproduced, reviewed, and converted into controlled answer vocabularies with deterministic grading and trajectory rubrics. Across 1,068 completed trajectories, the strongest model--harness pair passes 16/63 runs (25.4\%). scBench-Long evaluates whether agents can move beyond local analysis steps and make complex scientific claims that are supported by single-cell data.

Related papers

• SCTA: An Agentic Framework for Stable and Interpretable Target Gene Discovery from Single-Cell RNA Sequencing [31.966592797729888]
  We present SCTA (Single-Cell Target Agent), a decision-centric agentic framework for stable and interpretable target gene discovery from scRNA-seq data.<n>Rather than treating analysis as a single general-purpose reasoning task, SCTA decomposes target discovery into specialized agents aligned with key decision points in the single-cell pipeline.
  arXiv  Detail & Related papers  (2026-07-26T19:59:48Z)
• Benchmarking Machine Learning Models for Multi-Omics-Based Breast Cancer Prediction [0.0]
  Estrogen Receptor (ER) status is a critical biomarker in breast cancer diagnosis, prognosis, and treatment selection.<n>This study presents a systematic benchmarking analysis of classical machine learning models for ER status prediction.
  arXiv  Detail & Related papers  (2026-06-27T02:46:40Z)
• Influence-Guided Symbolic Regression: Scientific Discovery via LLM-Driven Equation Search with Granular Feedback [56.69850045068714]
  We introduce textitInfluence-Guided Regression (IGSR), a method that frames equation discovery as an iterative two-step process.<n>We demonstrate IGSRs effectiveness on a diverse suite of benchmarks, including LLM-SRBench, pharmacological PKPD models, an epidemiological simulation, and real-world genomic data.
  arXiv  Detail & Related papers  (2026-05-27T23:48:01Z)
• Verifiable Benchmarking of Long-Horizon Spatial Biology [0.0]
  SpatialBench-Long is a benchmark for long-horizon spatial biology.<n>It tests whether AI agents can move beyond executing procedural analysis to deriving accurate scientific conclusions from complex spatial measurements.<n>Three model-harness pairs tie at 8/72 runs (11.1%): Gemini 3.5 Flash / Pi terminal coding harness, GPT-5.5 / Pi, and GPT-5.5 / OpenAI Codex.
  arXiv  Detail & Related papers  (2026-05-27T07:19:18Z)
• RNA-FM: Flow-Matching Generative Model for Genome-wide RNA-Seq Prediction [48.07329582311906]
  We propose RNA-FM, a flow-matching generative framework for genome-wide bulk RNA-seq prediction from whole-slide images.<n> RNA-FM formulates transcriptomic prediction as a continuous-time conditional transport problem.<n>It enables scalable and biologically interpretable genome-wide gene expression imputation.
  arXiv  Detail & Related papers  (2026-05-12T06:48:58Z)
• ELISA: An Interpretable Hybrid Generative AI Agent for Expression-Grounded Discovery in Single-Cell Genomics [0.0]
  We introduce (Embedding-Linked Interactive Single-cell Agent) an interpretable framework that unifies scGPT expression embeddings with BioBERT-based semantic retrieval.<n> Benchmarked across six diverse scRNA-seq datasets spanning inflammatory lung disease, pediatric and adult cancers, organoid models, healthy tissue, and neurodevelopment.
  arXiv  Detail & Related papers  (2026-03-12T12:46:22Z)
• scMamba: A Pre-Trained Model for Single-Nucleus RNA Sequencing Analysis in Neurodegenerative Disorders [43.24785083027205]
  scMamba is a pre-trained model designed to improve the quality and utility of snRNA-seq analysis.<n>Inspired by the recent Mamba model, scMamba introduces a novel architecture that incorporates a linear adapter layer, gene embeddings, and bidirectional Mamba blocks.<n>We demonstrate that scMamba outperforms benchmark methods in various downstream tasks, including cell type annotation, doublet detection, imputation, and the identification of differentially expressed genes.
  arXiv  Detail & Related papers  (2025-02-12T11:48:22Z)
• GENERator: A Long-Context Generative Genomic Foundation Model [66.46537421135996]
  We present GENERator, a generative genomic foundation model featuring a context length of 98k base pairs (bp) and 1.2B parameters.<n>Trained on an expansive dataset comprising 386B bp of DNA, the GENERator demonstrates state-of-the-art performance across both established and newly proposed benchmarks.<n>It also shows significant promise in sequence optimization, particularly through the prompt-responsive generation of enhancer sequences with specific activity profiles.
  arXiv  Detail & Related papers  (2025-02-11T05:39:49Z)
• Graph Structure Learning for Tumor Microenvironment with Cell Type Annotation from non-spatial scRNA-seq data [6.432270457083369]
  We present a novel graph neural network (GNN) model that enhances cell type prediction and cell interaction analysis.<n>The proposed scGSL model demonstrated robust performance, achieving an average accuracy of 84.83%, precision of 86.23%, recall of 81.51%, and an F1 score of 80.92% across all datasets.
  arXiv  Detail & Related papers  (2025-02-04T18:28:25Z)
• BEACON: Benchmark for Comprehensive RNA Tasks and Language Models [60.02663015002029]
  We introduce the first comprehensive RNA benchmark BEACON (textbfBEnchmtextbfArk for textbfCOmprehensive RtextbfNA Task and Language Models).<n>First, BEACON comprises 13 distinct tasks derived from extensive previous work covering structural analysis, functional studies, and engineering applications.<n>Second, we examine a range of models, including traditional approaches like CNNs, as well as advanced RNA foundation models based on language models, offering valuable insights into the task-specific performances of these models.<n>Third, we investigate the vital RNA language model components
  arXiv  Detail & Related papers  (2024-06-14T19:39:19Z)
• scHyena: Foundation Model for Full-Length Single-Cell RNA-Seq Analysis in Brain [46.39828178736219]
  We introduce scHyena, a foundation model designed to address these challenges and enhance the accuracy of scRNA-seq analysis in the brain. scHyena is equipped with a linear adaptor layer, the positional encoding via gene-embedding, and a bidirectional Hyena operator. This enables us to process full-length scRNA-seq data without losing any information from the raw data.
  arXiv  Detail & Related papers  (2023-10-04T10:30:08Z)
• Predicting Molecular Phenotypes with Single Cell RNA Sequencing Data: an Assessment of Unsupervised Machine Learning Models [0.0]
  This study is to evaluate unsupervised machine learning on classifying treatment-resistant phenotypes in heterogeneous tumors. scRNAseq quantifies mRNA in cells and characterizes cell phenotypes. clusters generated from this pipeline can be used to understand cancer cell behavior and malignant growth.
  arXiv  Detail & Related papers  (2021-08-11T05:30:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.