A Cross Modal Knowledge Distillation & Data Augmentation Recipe for Improving Transcriptomics Representations through Morphological Features

• URL: http://arxiv.org/abs/2505.21317v1
• Date: Tue, 27 May 2025 15:15:34 GMT
• Title: A Cross Modal Knowledge Distillation & Data Augmentation Recipe for Improving Transcriptomics Representations through Morphological Features
• Authors: Ihab Bendidi, Yassir El Mesbahi, Alisandra K. Denton, Karush Suri, Kian Kenyon-Dean, Auguste Genovesio, Emmanuel Noutahi,
• Abstract summary: We propose a framework to enhance transcriptomics by distilling knowledge from microscopy images.<n>Using weakly paired data, our method aligns and binds modalities, enriching gene expression representations with morphological information.<n>These strategies improve the predictive power and retain the interpretability of transcriptomics, enabling rich unimodal representations for complex biological tasks.
• Score: 2.240470404069435
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Understanding cellular responses to stimuli is crucial for biological discovery and drug development. Transcriptomics provides interpretable, gene-level insights, while microscopy imaging offers rich predictive features but is harder to interpret. Weakly paired datasets, where samples share biological states, enable multimodal learning but are scarce, limiting their utility for training and multimodal inference. We propose a framework to enhance transcriptomics by distilling knowledge from microscopy images. Using weakly paired data, our method aligns and binds modalities, enriching gene expression representations with morphological information. To address data scarcity, we introduce (1) Semi-Clipped, an adaptation of CLIP for cross-modal distillation using pretrained foundation models, achieving state-of-the-art results, and (2) PEA (Perturbation Embedding Augmentation), a novel augmentation technique that enhances transcriptomics data while preserving inherent biological information. These strategies improve the predictive power and retain the interpretability of transcriptomics, enabling rich unimodal representations for complex biological tasks.

Related papers

• Towards Interpretable Drug-Drug Interaction Prediction: A Graph-Based Approach with Molecular and Network-Level Explanations [3.6099926707292793]
  Drug-drug interactions (DDIs) represent a critical challenge in pharmacology, often leading to adverse drug reactions with significant implications for patient safety and healthcare outcomes.<n>We propose MolecBioNet, a novel graph-based framework that integrates molecular and biomedical knowledge for robust and interpretable DDI prediction.
  arXiv  Detail & Related papers  (2025-07-12T07:43:19Z)
• Language-Enhanced Representation Learning for Single-Cell Transcriptomics [27.33236345953242]
  We propose scMMGPT, a novel framework designed for language-enhanced representation learning in single-cell transcriptomics.<n> scMMGPT employs robust cell representation extraction, preserving quantitative gene expression data, and introduces an innovative two-stage pre-training strategy.
  arXiv  Detail & Related papers  (2025-03-12T14:26:16Z)
• MVKTrans: Multi-View Knowledge Transfer for Robust Multiomics Classification [14.533025681231294]
  We propose the multi-view knowledge transfer learning framework, which transfers intra- and inter-omics knowledge in an adaptive manner. Specifically, we design a graph contrastive module that is trained on unlabeled data to effectively learn and transfer the underlying intra-omics patterns to the supervised task. In light of the varying discriminative capacities of modalities across different diseases and/or samples, we introduce an adaptive and bi-directional cross-omics distillation module.
  arXiv  Detail & Related papers  (2024-11-13T15:45:46Z)
• Causal Representation Learning from Multimodal Biomedical Observations [57.00712157758845]
  We develop flexible identification conditions for multimodal data and principled methods to facilitate the understanding of biomedical datasets.<n>Key theoretical contribution is the structural sparsity of causal connections between modalities.<n>Results on a real-world human phenotype dataset are consistent with established biomedical research.
  arXiv  Detail & Related papers  (2024-11-10T16:40:27Z)
• GTP-4o: Modality-prompted Heterogeneous Graph Learning for Omni-modal Biomedical Representation [68.63955715643974]
  Modality-prompted Heterogeneous Graph for Omnimodal Learning (GTP-4o) We propose an innovative Modality-prompted Heterogeneous Graph for Omnimodal Learning (GTP-4o)
  arXiv  Detail & Related papers  (2024-07-08T01:06:13Z)
• Weakly Supervised Set-Consistency Learning Improves Morphological Profiling of Single-Cell Images [0.6491172192043603]
  We propose a set-level consistency learning algorithm, Set-DINO, to improve learned representations of perturbation effects in single-cell images. We conduct experiments on a large-scale Optical Pooled Screening dataset with more than 5000 genetic perturbations.
  arXiv  Detail & Related papers  (2024-06-08T00:53:30Z)
• Tertiary Lymphoid Structures Generation through Graph-based Diffusion [54.37503714313661]
  In this work, we leverage state-of-the-art graph-based diffusion models to generate biologically meaningful cell-graphs. We show that the adopted graph diffusion model is able to accurately learn the distribution of cells in terms of their tertiary lymphoid structures (TLS) content.
  arXiv  Detail & Related papers  (2023-10-10T14:37:17Z)
• PathLDM: Text conditioned Latent Diffusion Model for Histopathology [62.970593674481414]
  We introduce PathLDM, the first text-conditioned Latent Diffusion Model tailored for generating high-quality histopathology images. Our approach fuses image and textual data to enhance the generation process. We achieved a SoTA FID score of 7.64 for text-to-image generation on the TCGA-BRCA dataset, significantly outperforming the closest text-conditioned competitor with FID 30.1.
  arXiv  Detail & Related papers  (2023-09-01T22:08:32Z)
• Modeling Dense Multimodal Interactions Between Biological Pathways and Histology for Survival Prediction [3.2274401541163322]
  We propose a memory-efficient multimodal Transformer that can model interactions between pathway and histology patch tokens. Our proposed model, SURVPATH, achieves state-of-the-art performance when evaluated against both unimodal and multimodal baselines.
  arXiv  Detail & Related papers  (2023-04-13T21:02:32Z)
• Learning multi-scale functional representations of proteins from single-cell microscopy data [77.34726150561087]
  We show that simple convolutional networks trained on localization classification can learn protein representations that encapsulate diverse functional information. We also propose a robust evaluation strategy to assess quality of protein representations across different scales of biological function.
  arXiv  Detail & Related papers  (2022-05-24T00:00:07Z)
• Evolution Is All You Need: Phylogenetic Augmentation for Contrastive Learning [1.7188280334580197]
  Self-supervised representation learning of biological sequence embeddings alleviates computational resource constraints on downstream tasks. We show that contrastive learning using evolutionary phylogenetic augmentation can be used as a representation learning objective.
  arXiv  Detail & Related papers  (2020-12-25T01:35:06Z)
• Towards an Automatic Analysis of CHO-K1 Suspension Growth in Microfluidic Single-cell Cultivation [63.94623495501023]
  We propose a novel Machine Learning architecture, which allows us to infuse a neural deep network with human-powered abstraction on the level of data. Specifically, we train a generative model simultaneously on natural and synthetic data, so that it learns a shared representation, from which a target variable, such as the cell count, can be reliably estimated.
  arXiv  Detail & Related papers  (2020-10-20T08:36:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.