RankByGene: Gene-Guided Histopathology Representation Learning Through Cross-Modal Ranking Consistency

• URL: http://arxiv.org/abs/2411.15076v1
• Date: Fri, 22 Nov 2024 17:08:28 GMT
• Title: RankByGene: Gene-Guided Histopathology Representation Learning Through Cross-Modal Ranking Consistency
• Authors: Wentao Huang, Meilong Xu, Xiaoling Hu, Shahira Abousamra, Aniruddha Ganguly, Saarthak Kapse, Alisa Yurovsky, Prateek Prasanna, Tahsin Kurc, Joel Saltz, Michael L. Miller, Chao Chen,
• Abstract summary: We propose a novel framework that aligns gene and image features using a ranking-based alignment loss. To further enhance the alignment's stability, we employ self-supervised knowledge distillation with a teacher-student network architecture.
• Score: 11.813883157319381
• License:
• Abstract: Spatial transcriptomics (ST) provides essential spatial context by mapping gene expression within tissue, enabling detailed study of cellular heterogeneity and tissue organization. However, aligning ST data with histology images poses challenges due to inherent spatial distortions and modality-specific variations. Existing methods largely rely on direct alignment, which often fails to capture complex cross-modal relationships. To address these limitations, we propose a novel framework that aligns gene and image features using a ranking-based alignment loss, preserving relative similarity across modalities and enabling robust multi-scale alignment. To further enhance the alignment's stability, we employ self-supervised knowledge distillation with a teacher-student network architecture, effectively mitigating disruptions from high dimensionality, sparsity, and noise in gene expression data. Extensive experiments on gene expression prediction and survival analysis demonstrate our framework's effectiveness, showing improved alignment and predictive performance over existing methods and establishing a robust tool for gene-guided image representation learning in digital pathology.

Related papers

• Unsupervised Feature Orthogonalization for Learning Distortion-Invariant Representations [0.13108652488669734]
  This study introduces unORANIC+, a novel method that integrates unsupervised feature orthogonalization with the ability of a Vision Transformer. The streamlined architecture of unORANIC+ effectively separates anatomical and image-specific attributes, resulting in robust and unbiased latent representations. Extensive experimentation demonstrates unORANIC+'s reconstruction proficiency, corruption resilience, as well as capability to revise existing image distortions. We confirm its adaptability to diverse datasets of varying image sources and sample sizes which positions the method as a promising algorithm for advanced medical image analysis.
  arXiv  Detail & Related papers  (2024-09-18T19:25:38Z)
• Distance-Preserving Generative Modeling of Spatial Transcriptomics [0.0]
  We introduce a class of distance-preserving generative models for spatial transcriptomics. We use the provided spatial information to regularize the learned representation space of gene expressions to have a similar pair-wise distance structure. Our framework grants compatibility with any variational-inference-based generative models for gene expression modeling.
  arXiv  Detail & Related papers  (2024-08-01T21:04:27Z)
• Revisiting Adaptive Cellular Recognition Under Domain Shifts: A Contextual Correspondence View [49.03501451546763]
  We identify the importance of implicit correspondences across biological contexts for exploiting domain-invariant pathological composition. We propose self-adaptive dynamic distillation to secure instance-aware trade-offs across different model constituents.
  arXiv  Detail & Related papers  (2024-07-14T04:41:16Z)
• Spatially Resolved Gene Expression Prediction from Histology via Multi-view Graph Contrastive Learning with HSIC-bottleneck Regularization [18.554968935341236]
  We propose a Multi-view Graph Contrastive Learning framework with HSIC-bottleneck Regularization(ST-GCHB) to help impute the gene expression of the queried imagingspots by considering their spatial dependency.
  arXiv  Detail & Related papers  (2024-06-18T03:07:25Z)
• Weakly Supervised Set-Consistency Learning Improves Morphological Profiling of Single-Cell Images [0.6491172192043603]
  We propose a set-level consistency learning algorithm, Set-DINO, to improve learned representations of perturbation effects in single-cell images. We conduct experiments on a large-scale Optical Pooled Screening dataset with more than 5000 genetic perturbations.
  arXiv  Detail & Related papers  (2024-06-08T00:53:30Z)
• Genetic InfoMax: Exploring Mutual Information Maximization in High-Dimensional Imaging Genetics Studies [50.11449968854487]
  Genome-wide association studies (GWAS) are used to identify relationships between genetic variations and specific traits. Representation learning for imaging genetics is largely under-explored due to the unique challenges posed by GWAS. We introduce a trans-modal learning framework Genetic InfoMax (GIM) to address the specific challenges of GWAS.
  arXiv  Detail & Related papers  (2023-09-26T03:59:21Z)
• Spatially Resolved Gene Expression Prediction from H&E Histology Images via Bi-modal Contrastive Learning [4.067498002241427]
  We present BLEEP (Bi-modaL Embedding for Expression Prediction), a bi-modal embedding framework capable of generating spatially resolved gene expression profiles. BLEEP uses contrastive learning to construct a low-dimensional joint embedding space from a reference dataset using paired image and expression profiles at micrometer resolution. We demonstrate BLEEP's effectiveness in gene expression prediction by benchmarking its performance on a human liver tissue dataset captured using the 10x Visium platform.
  arXiv  Detail & Related papers  (2023-06-02T18:27:26Z)
• Unsupervised Domain Transfer with Conditional Invertible Neural Networks [83.90291882730925]
  We propose a domain transfer approach based on conditional invertible neural networks (cINNs) Our method inherently guarantees cycle consistency through its invertible architecture, and network training can efficiently be conducted with maximum likelihood. Our method enables the generation of realistic spectral data and outperforms the state of the art on two downstream classification tasks.
  arXiv  Detail & Related papers  (2023-03-17T18:00:27Z)
• Bridging Synthetic and Real Images: a Transferable and Multiple Consistency aided Fundus Image Enhancement Framework [61.74188977009786]
  We propose an end-to-end optimized teacher-student framework to simultaneously conduct image enhancement and domain adaptation. We also propose a novel multi-stage multi-attention guided enhancement network (MAGE-Net) as the backbones of our teacher and student network.
  arXiv  Detail & Related papers  (2023-02-23T06:16:15Z)
• Unsupervised ensemble-based phenotyping helps enhance the discoverability of genes related to heart morphology [57.25098075813054]
  We propose a new framework for gene discovery entitled Un Phenotype Ensembles. It builds a redundant yet highly expressive representation by pooling a set of phenotypes learned in an unsupervised manner. These phenotypes are then analyzed via (GWAS), retaining only highly confident and stable associations.
  arXiv  Detail & Related papers  (2023-01-07T18:36:44Z)
• Pathological Retinal Region Segmentation From OCT Images Using Geometric Relation Based Augmentation [84.7571086566595]
  We propose improvements over previous GAN-based medical image synthesis methods by jointly encoding the intrinsic relationship of geometry and shape. The proposed method outperforms state-of-the-art segmentation methods on the public RETOUCH dataset having images captured from different acquisition procedures.
  arXiv  Detail & Related papers  (2020-03-31T11:50:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.