Spatial Transcriptomics as Images for Large-Scale Pretraining

• URL: http://arxiv.org/abs/2603.13432v2
• Date: Wed, 18 Mar 2026 04:20:33 GMT
• Title: Spatial Transcriptomics as Images for Large-Scale Pretraining
• Abstract summary: Spatial Transcriptomics profiles thousands of gene expression values at discrete spots with precise coordinates on tissue sections.<n>With rising sequencing throughput and advancing platforms, the expanding data volumes motivate large-scale ST pretraining.<n>Existing choices fall into two camps: (1) treating each spot as an independent sample, which discards spatial dependencies and collapses ST into single-cell transcriptomics; and (2) treating an entire slide as a single sample, which produces prohibitively large inputs and drastically fewer training examples.<n>We propose treating spatial transcriptomics as croppable images. Specifically, we define a multi-channel image representation with fixed spatial size by cropping patches
• Score: 18.7187666986181
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Spatial Transcriptomics (ST) profiles thousands of gene expression values at discrete spots with precise coordinates on tissue sections, preserving spatial context essential for clinical and pathological studies. With rising sequencing throughput and advancing platforms, the expanding data volumes motivate large-scale ST pretraining. However, the fundamental unit for pretraining, i.e., what constitutes a single training sample, remains ill-posed. Existing choices fall into two camps: (1) treating each spot as an independent sample, which discards spatial dependencies and collapses ST into single-cell transcriptomics; and (2) treating an entire slide as a single sample, which produces prohibitively large inputs and drastically fewer training examples, undermining effective pretraining. To address this gap, we propose treating spatial transcriptomics as croppable images. Specifically, we define a multi-channel image representation with fixed spatial size by cropping patches from raw slides, thereby preserving spatial context while substantially increasing the number of training samples. Along the channel dimension, we define gene subset selection rules to control input dimensionality and improve pretraining stability. Extensive experiments show that the proposed image-like dataset construction for ST pretraining consistently improves downstream performance, outperforming conventional pretraining schemes. Ablation studies verify that both spatial patching and channel design are necessary, establishing a unified, practical paradigm for organizing ST data and enabling large-scale pretraining.

Related papers

• Self-Supervised Representation-Guided Generative Dataset Distillation [55.27114962330541]
  We propose self-supervised representation-guided generative dataset distillation (SRG)<n>SRG translates the SSL geometry into diffusion guidance.<n>It consistently outperforms evaluated generative baselines across multiple datasets and IPC settings.
  arXiv  Detail & Related papers  (2026-08-04T06:51:45Z)
• GRASP: Geometry-aware Residual Alignment for Scalable Pretraining Data Attribution [5.770893169582546]
  This work reframes attribution as subset-level counterfactual utility prediction.<n>We introduce GRASP, an interaction-aware surrogate.<n>It decisively outperforms existing scalable baselines.
  arXiv  Detail & Related papers  (2026-06-05T04:17:50Z)
• Learning to Synergize Semantic and Geometric Priors for Limited-Data Wheat Disease Segmentation [51.06355116973389]
  We propose a framework that treats wheat disease segmentation as a coupled task of disease-specific semantic perception and disease boundary localization.<n>Our core insight is that pretrained DINOv2 provides robust category-aware semantic priors to handle appearance shifts.<n>We show that SGPer consistently achieves state-of-the-art performance on wheat disease and organ segmentation benchmarks.
  arXiv  Detail & Related papers  (2026-04-07T04:19:39Z)
• Rare-Aware Autoencoding: Reconstructing Spatially Imbalanced Data [49.13338489780734]
  Autoencoders can be challenged by spatially non-uniform sampling of image content.<n>In practice, autoencoders are biased toward dominant patterns resulting in the loss of fine-grained detail.<n>We address spatial imbalance by two complementary components: (i) self-entropy-based loss that upweights statistically uncommon spatial locations.
  arXiv  Detail & Related papers  (2026-04-02T13:38:29Z)
• Fake It Right: Injecting Anatomical Logic into Synthetic Supervised Pre-training for Medical Segmentation [21.75204301463342]
  Vision Transformers (ViTs) excel in 3D medical segmentation but require massive datasets.<n>Formula-Driven Supervised Learning (F) offers a privacy-preserving alternative by pre-training on synthetic mathematical primitives.<n>We propose an annotated-Informed Synthetic Anatomy Supervised Pre-training framework unifying F's infinite scalability with anatomical realism.
  arXiv  Detail & Related papers  (2026-03-01T08:15:18Z)
• Efficient Conformal Volumetry for Template-Based Segmentation [23.593891761167374]
  We introduce ConVOLT, a framework that achieves efficient volumetric uncertainty quantification.<n>ConVOLT calibrates a learned volumetric scaling factor from deformation space features.<n>Our work paves way to exploit the registration process for efficient UQ in medical imaging pipelines.
  arXiv  Detail & Related papers  (2026-02-28T20:16:15Z)
• Learning to Factorize and Adapt: A Versatile Approach Toward Universal Spatio-Temporal Foundation Models [42.152122602443164]
  We present FactoST-v2, an enhanced factorized framework for universal temporal learning.<n>We show that FactoST-v2 achieves state-of-the-art accuracy with linear efficiency.<n>This factorized paradigm offers a practical, scalable path toward truly universal STFMs.
  arXiv  Detail & Related papers  (2026-01-17T15:20:08Z)
• Data Efficiency and Transfer Robustness in Biomedical Image Segmentation: A Study of Redundancy and Forgetting with Cellpose [2.37771211295026]
  Generalist biomedical image segmentation models such as Cellpose are increasingly applied across diverse imaging modalities and cell types.<n>In this study, we conduct a systematic empirical analysis of challenges using Cellpose as a case study.<n>To assess data redundancy, we propose a simple dataset quantization (DQ) strategy for constructing compact yet diverse training subsets.<n>To examine catastrophic forgetting, we perform cross domain finetuning experiments and observe significant degradation in source domain performance.
  arXiv  Detail & Related papers  (2025-11-06T20:45:56Z)
• ST-DAI: Single-shot 2.5D Spatial Transcriptomics with Intra-Sample Domain Adaptive Imputation for Cost-efficient 3D Reconstruction [1.7603474309877931]
  We introduce ST-DAI, a single-shot framework for 3D transcriptomics that couples a cost-efficient 2.5D sampling scheme with an intra-sample domain-adaptive imputation framework.<n>First, in the cost-efficient 2.5D sampling stage, one reference section (central section) is fully sampled while other sections (adjacent sections) are sparsely sampled.<n>Second, we propose a single-shot 3D imputation learning method that allows us to generate fully sampled 3D ST from this cost-efficient 2.5D ST scheme.
  arXiv  Detail & Related papers  (2025-07-29T05:46:37Z)
• A Partial Initialization Strategy to Mitigate the Overfitting Problem in CATE Estimation with Hidden Confounding [44.874826691991565]
  Estimating the conditional average treatment effect (CATE) from observational data plays a crucial role in areas such as e-commerce, healthcare, and economics.<n>Existing studies mainly rely on the strong ignorability assumption that there are no hidden confounders.<n>Data collected from randomized controlled trials (RCT) do not suffer from confounding but are usually limited by a small sample size.
  arXiv  Detail & Related papers  (2025-01-15T15:58:16Z)
• Downstream-Pretext Domain Knowledge Traceback for Active Learning [138.02530777915362]
  We propose a downstream-pretext domain knowledge traceback (DOKT) method that traces the data interactions of downstream knowledge and pre-training guidance. DOKT consists of a traceback diversity indicator and a domain-based uncertainty estimator. Experiments conducted on ten datasets show that our model outperforms other state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-07-20T01:34:13Z)
• Prior Knowledge-Guided Attention in Self-Supervised Vision Transformers [79.60022233109397]
  We present spatial prior attention (SPAN), a framework that takes advantage of consistent spatial and semantic structure in unlabeled image datasets. SPAN operates by regularizing attention masks from separate transformer heads to follow various priors over semantic regions. We find that the resulting attention masks are more interpretable than those derived from domain-agnostic pretraining.
  arXiv  Detail & Related papers  (2022-09-07T02:30:36Z)
• On the Viability of Monocular Depth Pre-training for Semantic Segmentation [48.29060171161375]
  We study whether pre-training on geometric tasks is viable for downstream transfer to semantic tasks. We find that monocular depth is a viable form of pre-training for semantic segmentation, validated by improvements over common baselines.
  arXiv  Detail & Related papers  (2022-03-26T04:27:28Z)
• Dense Contrastive Visual-Linguistic Pretraining [53.61233531733243]
  Several multimodal representation learning approaches have been proposed that jointly represent image and text. These approaches achieve superior performance by capturing high-level semantic information from large-scale multimodal pretraining. We propose unbiased Dense Contrastive Visual-Linguistic Pretraining to replace the region regression and classification with cross-modality region contrastive learning.
  arXiv  Detail & Related papers  (2021-09-24T07:20:13Z)
• Context-self contrastive pretraining for crop type semantic segmentation [39.81074867563505]
  The proposed Context-Self Contrastive Loss (CSCL) learns an embedding space that makes semantic boundaries pop-up. For crop type semantic segmentation from Satellite Image Time Series (SITS) we find performance at parcel boundaries to be a critical bottleneck. We present a process for semantic segmentation at super-resolution for obtaining crop classes at a more granular level.
  arXiv  Detail & Related papers  (2021-04-09T11:29:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.