Score Distributions, Not Cells: Evaluating Single-Cell Perturbations Under Class Overlap

• URL: http://arxiv.org/abs/2607.04595v1
• Date: Mon, 06 Jul 2026 01:47:02 GMT
• Title: Score Distributions, Not Cells: Evaluating Single-Cell Perturbations Under Class Overlap
• Abstract summary: Two perturbations can produce different populations of cells while overlapping so much that an individual cell could belong to either.<n>Per-cell accuracy measures this overlap rather than model quality.<n>The fix is to score perturbations across the whole population rather than cell by cell.
• Score: 0.09558392439655013
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Most classification problems assume the classes are roughly separable, so that an individual sample can usually be assigned to one class. Single-cell perturbation data violates this assumption: two perturbations can produce different populations of cells while overlapping so much that an individual cell could belong to either. Per-cell accuracy then measures this overlap rather than model quality. We see this on Tahoe-100M and the Virtual Cell Challenge, where a linear classifier, an MLP, and a Transformer all plateau near macro-F1 0.2-0.3 even though almost every pair of perturbations is statistically distinguishable. The fix is to score perturbations across the whole population rather than cell by cell. We average a classifier's per-cell probability vectors over all cells of a perturbation to form a population profile, then rank candidate perturbations by this profile; we call the resulting score the Classifier Discrimination Score (CDS). Taking the top-ranked class recovers the winning perturbation. It needs no retraining, costs linear time in the number of cells, and recovers near-perfect identification from the same weak models. CDS differs from the pseudobulk-based Perturbation Discrimination Score (PDS) used in recent benchmarks only in where the average is taken, raw gene expression for PDS versus a learned discriminative space for CDS, and identifies the true perturbation more reliably on both datasets, with the gap widening as cells grow scarce. Because a metric that misranks the ground truth will misrank the models scored against it, per-cell accuracy and raw-pseudobulk scores should be used with caution when comparing perturbation models.

Related papers

• PopPert: Population-level Joint-Distribution Modeling for Single-Cell Perturbation Prediction [17.491072730946158]
  We propose PopPert, a framework that explicitly parameterizes population-level joint gene expression distributions.<n>Given a control population distribution and a perturbation condition, PopPert predicts perturbation-induced changes in distribution parameters.<n>PopPert achieves superior overall performance in differential expression recovery, perturbation effect estimation, and population-level distribution matching.
  arXiv  Detail & Related papers  (2026-09-01T14:59:03Z)
• PerturbCellRL: Verifier-Guided Reinforcement Learning for Single-Cell Perturbation Prediction [28.013973806760976]
  Single-cell perturbation models can reduce costly wet-lab screening by predicting how cells respond to interventions.<n>We introduce PerturbCellRL, a reinforcement learning framework that post-trains a pretrained single-cell transcriptomic generator.<n>We evaluate PerturbCellRL on multiple genetic and chemical perturbation benchmarks.
  arXiv  Detail & Related papers  (2026-06-26T06:15:04Z)
• MAT-Cell: A Multi-Agent Tree-Structured Reasoning Framework for Batch-Level Single-Cell Annotation [81.82645086402366]
  We propose MAT-Cell, a neuro-symbolic reasoning framework that reframes single-cell analysis into constructive, verifiable proof generation.<n> MAT-Cell injects symbolic constraints through adaptive Retrieval-Augmented Generation (RAG) to ground neural reasoning in biological axioms and reduce transcriptomic noise.<n>It significantly outperforms state-of-the-art (SOTA) models and maintains robust per-formance in challenging scenarios.
  arXiv  Detail & Related papers  (2026-04-07T04:56:06Z)
• PerturbDiff: Functional Diffusion for Single-Cell Perturbation Modeling [32.464109652332915]
  We introduce PerturbDiff, which shifts modeling from individual cells to entire distributions.<n>PerturbDiff achieves state-of-the-art performance in single-cell response prediction and generalizes substantially better to unseen perturbations.
  arXiv  Detail & Related papers  (2026-02-23T10:28:56Z)
• Departures: Distributional Transport for Single-Cell Perturbation Prediction with Neural Schrödinger Bridges [51.83259180910313]
  A major bottleneck in gene function analysis is the unpaired nature of single-cell data.<n>We approximate Schrdinger Bridge (SB) to tackle unpaired single-cell perturbation data.<n>Our model effectively captures heterogeneous single-cell responses and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2025-11-17T08:27:13Z)
• Unlasting: Unpaired Single-Cell Multi-Perturbation Estimation by Dual Conditional Diffusion Implicit Bridges [68.98973318553983]
  We propose a framework based on Dual Diffusion Implicit Bridges (DDIB) to learn the mapping between different data distributions.<n>We integrate gene regulatory network (GRN) information to propagate perturbation signals in a biologically meaningful way.<n>We also incorporate a masking mechanism to predict silent genes, improving the quality of generated profiles.
  arXiv  Detail & Related papers  (2025-06-26T09:05:38Z)
• Learnable Distribution Calibration for Few-Shot Class-Incremental Learning [122.2241120474278]
  Few-shot class-incremental learning (FSCIL) faces challenges of memorizing old class distributions and estimating new class distributions given few training samples. We propose a learnable distribution calibration (LDC) approach, with the aim to systematically solve these two challenges using a unified framework.
  arXiv  Detail & Related papers  (2022-10-01T09:40:26Z)
• The Cellwise Minimum Covariance Determinant Estimator [1.90365714903665]
  We propose a cellwise robust version of the MCD method, called cellMCD. It performs well in simulations with cellwise outliers, and has high finite-sample efficiency on clean data. It is illustrated with real data with visualizations of the results.
  arXiv  Detail & Related papers  (2022-07-27T12:33:51Z)
• PLM: Partial Label Masking for Imbalanced Multi-label Classification [59.68444804243782]
  Neural networks trained on real-world datasets with long-tailed label distributions are biased towards frequent classes and perform poorly on infrequent classes. We propose a method, Partial Label Masking (PLM), which utilizes this ratio during training. Our method achieves strong performance when compared to existing methods on both multi-label (MultiMNIST and MSCOCO) and single-label (imbalanced CIFAR-10 and CIFAR-100) image classification datasets.
  arXiv  Detail & Related papers  (2021-05-22T18:07:56Z)
• Red Blood Cell Segmentation with Overlapping Cell Separation and Classification on Imbalanced Dataset [1.7219362335740878]
  Overlapping cells can cause incorrect predicted results that have to separate into multiple single RBCs before classifying. To classify multiple classes with deep learning, imbalance problems are common in medical imaging because normal samples are always higher than rare disease samples. This paper presents a new method to segment and classify red blood cells from blood smear images, specifically to tackle cell overlapping and data imbalance problems.
  arXiv  Detail & Related papers  (2020-12-02T16:49:51Z)
• Selective Classification Can Magnify Disparities Across Groups [89.14499988774985]
  We find that while selective classification can improve average accuracies, it can simultaneously magnify existing accuracy disparities. Increasing abstentions can even decrease accuracies on some groups. We train distributionally-robust models that achieve similar full-coverage accuracies across groups and show that selective classification uniformly improves each group.
  arXiv  Detail & Related papers  (2020-10-27T08:51:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.