Exemplar Guided Deep Neural Network for Spatial Transcriptomics Analysis of Gene Expression Prediction

• URL: http://arxiv.org/abs/2210.16721v1
• Date: Sun, 30 Oct 2022 02:22:20 GMT
• Title: Exemplar Guided Deep Neural Network for Spatial Transcriptomics Analysis of Gene Expression Prediction
• Abstract summary: This paper proposes an Exemplar Guided Network (EGN) to accurately and efficiently predict gene expression directly from each window of a tissue slide image. Our EGN framework composes of three main components: 1) an extractor to structure a representation space for unsupervised retrievals; 2) a vision transformer (ViT) backbone to progressively extract representations of the input window; and 3) an Exemplar Bridging (EB) block to adaptively revise the intermediate ViT representations by using the nearest exemplars.
• Score: 9.192169460752805
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Spatial transcriptomics (ST) is essential for understanding diseases and developing novel treatments. It measures gene expression of each fine-grained area (i.e., different windows) in the tissue slide with low throughput. This paper proposes an Exemplar Guided Network (EGN) to accurately and efficiently predict gene expression directly from each window of a tissue slide image. We apply exemplar learning to dynamically boost gene expression prediction from nearest/similar exemplars of a given tissue slide image window. Our EGN framework composes of three main components: 1) an extractor to structure a representation space for unsupervised exemplar retrievals; 2) a vision transformer (ViT) backbone to progressively extract representations of the input window; and 3) an Exemplar Bridging (EB) block to adaptively revise the intermediate ViT representations by using the nearest exemplars. Finally, we complete the gene expression prediction task with a simple attention-based prediction block. Experiments on standard benchmark datasets indicate the superiority of our approach when comparing with the past state-of-the-art (SOTA) methods.

Related papers

• Generative Retrieval for Unsupervised Text-Based Person Search [77.46429603553388]
  Text-based person search (TBPS) aims to retrieve images of a target person from a large image gallery based on a given natural language description.<n>We propose GTR+, a two-stage generation-then-retrieval framework.<n>We also contribute LargeFine-Person, a large-scale TBPS dataset with high-quality, fine-grained, and diverse textual annotations.
  arXiv  Detail & Related papers  (2026-09-11T15:25:00Z)
• Rethinking Message Passing as Retrieval for Text-Attributed Graph Learning [63.602658976990874]
  Graph neural networks (GNNs) are typically conceptualized as message-passing neural networks.<n>We present a retrieval-augmented view of GNNs: each layer makes predictions by applying anvariant to a node representation together with a permutation-in summary of retrieved graph context.<n>Motivated by this perspective, we propose RTA, a simple aggregation-based framework that replaces structural message passing with label-aware retrieval and propagation.
  arXiv  Detail & Related papers  (2026-08-27T07:25:44Z)
• Contrastive and Adaptive Multi-modal Masked Autoencoder for Spatial Transcriptomics [9.067087306213525]
  We use partial gene expression to guide the prediction process alongside histology images.<n>We employ a Masked Autoencoder (MAE) to utilize a small fraction of gene expression as genetic anchors for inferring whole-slide gene expression profiles.<n>Our framework consistently surpasses existing methods in both histology-only prediction and spatial imputation.
  arXiv  Detail & Related papers  (2026-06-19T06:47:55Z)
• Reward-Guided Semantic Evolution for Test-time Adaptive Object Detection [82.2968697030677]
  Open-vocabulary object detection with vision-language models (VLMs) such as Grounding DINO suffers from performance degradation under test-time distribution shifts.<n>We propose Reward-Guided Semantic Evolution (RGSE), a training-free framework that directly refines the text embeddings at test time.
  arXiv  Detail & Related papers  (2026-05-06T06:17:41Z)
• GenAR: Next-Scale Autoregressive Generation for Spatial Gene Expression Prediction [15.143858141542532]
  GenAR is a multi-scale autoregressive framework that refines predictions from coarse to fine.<n>We introduce GenAR, a multi-scale autoregressive framework that refines predictions from coarse to fine.<n>GenAR achieves principled state-of-the-art performance, offering potential implications for precision medicine and cost-effective molecular profiling.
  arXiv  Detail & Related papers  (2025-10-05T18:28:21Z)
• Learning from Gene Names, Expression Values and Images: Contrastive Masked Text-Image Pretraining for Spatial Transcriptomics Representation Learning [2.234171233882506]
  CoMTIP is a Contrastive Masked Text-Image Pretraining framework.<n>It learns from images, gene names, and expression values while capturing fine-grained visual context for spatial transcriptomics.<n>CoMTIP not only surpasses previous methods on diverse downstream tasks but also achieves zero-shot gene expression prediction.
  arXiv  Detail & Related papers  (2025-09-21T03:05:17Z)
• PRISM-0: A Predicate-Rich Scene Graph Generation Framework for Zero-Shot Open-Vocabulary Tasks [51.31903029903904]
  In Scene Graphs Generation (SGG) one extracts structured representation from visual inputs in the form of objects nodes and predicates connecting them. PRISM-0 is a framework for zero-shot open-vocabulary SGG that bootstraps foundation models in a bottom-up approach. PRIMS-0 generates semantically meaningful graphs that improve downstream tasks such as Image Captioning and Sentence-to-Graph Retrieval.
  arXiv  Detail & Related papers  (2025-04-01T14:29:51Z)
• Boundary-Guided Learning for Gene Expression Prediction in Spatial Transcriptomics [7.763803040383128]
  We propose a framework named BG-TRIPLEX, which leverages boundary information extracted from pathological images as guiding features to enhance gene expression prediction. Our framework consistently outperforms existing methods in terms of Pearson Correlation Coefficient (PCC) This method highlights the crucial role of boundary features in understanding the complex interactions between WSI and gene expression.
  arXiv  Detail & Related papers  (2024-12-05T11:09:11Z)
• MERGE: Multi-faceted Hierarchical Graph-based GNN for Gene Expression Prediction from Whole Slide Histopathology Images [6.717786190771243]
  We introduce MERGE (Multi-faceted hiErarchical gRaph for Gene Expressions), which combines a hierarchical graph construction strategy with graph neural networks (GNN) to improve gene expression predictions from whole slide images. By clustering tissue image patches based on both spatial and morphological features, our approach fosters interactions between distant tissue locations during GNN learning. As an additional contribution, we evaluate different data smoothing techniques that are necessary to mitigate artifacts in ST data, often caused by technical imperfections.
  arXiv  Detail & Related papers  (2024-12-03T17:32:05Z)
• Interpretable Next-token Prediction via the Generalized Induction Head [59.500195503897764]
  Generalized Induction-Head Model (GIM) is an interpretable model for next-token prediction.<n>In language modeling, GIM improves next-token prediction by up to 25%p over interpretable baselines.<n>In an fMRI setting, GIM improves neural response prediction by 20%.
  arXiv  Detail & Related papers  (2024-10-31T12:33:26Z)
• SpaRED benchmark: Enhancing Gene Expression Prediction from Histology Images with Spatial Transcriptomics Completion [2.032350440475489]
  We present a systematically curated and processed database collected from 26 public sources. We also propose a state-of-the-art transformer based completion technique for inferring missing gene expression. Our contributions constitute the most comprehensive benchmark of gene expression prediction from histology images to date.
  arXiv  Detail & Related papers  (2024-07-17T21:28:20Z)
• S^2Former-OR: Single-Stage Bi-Modal Transformer for Scene Graph Generation in OR [50.435592120607815]
  Scene graph generation (SGG) of surgical procedures is crucial in enhancing holistically cognitive intelligence in the operating room (OR) Previous works have primarily relied on multi-stage learning, where the generated semantic scene graphs depend on intermediate processes with pose estimation and object detection. In this study, we introduce a novel single-stage bi-modal transformer framework for SGG in the OR, termed S2Former-OR.
  arXiv  Detail & Related papers  (2024-02-22T11:40:49Z)
• Spatial Transcriptomics Analysis of Zero-shot Gene Expression Prediction [7.8979634764500455]
  We propose a pioneering zero-shot framework for predicting gene expression from slide image windows. Considering a gene type can be described by functionality and phenotype, we dynamically embed a gene type to a vector. We employ this vector to project slide image windows to gene expression in feature space, unleashing zero-shot expression prediction for unseen gene types.
  arXiv  Detail & Related papers  (2024-01-26T10:53:21Z)
• Forgery-aware Adaptive Transformer for Generalizable Synthetic Image Detection [106.39544368711427]
  We study the problem of generalizable synthetic image detection, aiming to detect forgery images from diverse generative methods. We present a novel forgery-aware adaptive transformer approach, namely FatFormer. Our approach tuned on 4-class ProGAN data attains an average of 98% accuracy to unseen GANs, and surprisingly generalizes to unseen diffusion models with 95% accuracy.
  arXiv  Detail & Related papers  (2023-12-27T17:36:32Z)
• Leveraging Graph Diffusion Models for Network Refinement Tasks [72.54590628084178]
  We propose a novel graph generative framework, SGDM, based on subgraph diffusion. Our framework not only improves the scalability and fidelity of graph diffusion models, but also leverages the reverse process to perform novel, conditional generation tasks.
  arXiv  Detail & Related papers  (2023-11-29T18:02:29Z)
• SEPAL: Spatial Gene Expression Prediction from Local Graphs [1.4523812806185954]
  We present SEPAL, a new model for predicting genetic profiles from visual tissue appearance. Our method exploits the biological biases of the problem by directly supervising relative differences with respect to mean expression. We propose a novel benchmark that aims to better define the task by following current best practices in transcriptomics.
  arXiv  Detail & Related papers  (2023-09-02T23:24:02Z)
• Text Enriched Sparse Hyperbolic Graph Convolutional Networks [21.83127488157701]
  Graph Neural Networks (GNNs) and their hyperbolic variants provide a promising approach to encode such networks in a low-dimensional latent space. We propose Text Enriched Sparse Hyperbolic Graph Convolution Network (TESH-GCN) to capture the graph's metapath structures using semantic signals. Our model outperforms the current state-of-the-art approaches by a large margin on the task of link prediction.
  arXiv  Detail & Related papers  (2022-07-06T00:23:35Z)
• Two-Stream Graph Convolutional Network for Intra-oral Scanner Image Segmentation [133.02190910009384]
  We propose a two-stream graph convolutional network (i.e., TSGCN) to handle inter-view confusion between different raw attributes. Our TSGCN significantly outperforms state-of-the-art methods in 3D tooth (surface) segmentation.
  arXiv  Detail & Related papers  (2022-04-19T10:41:09Z)
• HETFORMER: Heterogeneous Transformer with Sparse Attention for Long-Text Extractive Summarization [57.798070356553936]
  HETFORMER is a Transformer-based pre-trained model with multi-granularity sparse attentions for extractive summarization. Experiments on both single- and multi-document summarization tasks show that HETFORMER achieves state-of-the-art performance in Rouge F1.
  arXiv  Detail & Related papers  (2021-10-12T22:42:31Z)
• Adaptive Bi-directional Attention: Exploring Multi-Granularity Representations for Machine Reading Comprehension [29.717816161964105]
  We propose a novel approach called Adaptive Bidirectional Attention, which adaptively exploits the source representations of different levels to the predictor. Results are better than the previous state-of-the-art model by 2.5$%$ EM and 2.3$%$ F1 scores.
  arXiv  Detail & Related papers  (2020-12-20T09:31:35Z)
• Select, Extract and Generate: Neural Keyphrase Generation with Layer-wise Coverage Attention [75.44523978180317]
  We propose emphSEG-Net, a neural keyphrase generation model that is composed of two major components. The experimental results on seven keyphrase generation benchmarks from scientific and web documents demonstrate that SEG-Net outperforms the state-of-the-art neural generative methods by a large margin.
  arXiv  Detail & Related papers  (2020-08-04T18:00:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.