Momentum Memory for Knowledge Distillation in Computational Pathology

• URL: http://arxiv.org/abs/2602.21395v2
• Date: Wed, 04 Mar 2026 14:47:14 GMT
• Title: Momentum Memory for Knowledge Distillation in Computational Pathology
• Abstract summary: We propose Momentum Memory Knowledge Distillation (MoMKD), a cross-modal distillation framework driven by a momentum-updated memory.<n>MoMKD consistently outperforms state-of-the-art MIL and multimodal KD baselines, delivering strong performance and generalization under histology-only inference.
• Score: 12.305907179979426
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multimodal learning that integrates genomics and histopathology has shown strong potential in cancer diagnosis, yet its clinical translation is hindered by the limited availability of paired histology-genomics data. Knowledge distillation (KD) offers a practical solution by transferring genomic supervision into histopathology models, enabling accurate inference using histology alone. However, existing KD methods rely on batch-local alignment, which introduces instability due to limited within-batch comparisons and ultimately degrades performance. To address these limitations, we propose Momentum Memory Knowledge Distillation (MoMKD), a cross-modal distillation framework driven by a momentum-updated memory. This memory aggregates genomic and histopathology information across batches, effectively enlarging the supervisory context available to each mini-batch. Furthermore, we decouple the gradients of the genomics and histology branches, preventing genomic signals from dominating histology feature learning during training and eliminating the modality-gap issue at inference time. Extensive experiments on the TCGA-BRCA benchmark (HER2, PR, and ODX classification tasks) and an independent in-house testing dataset demonstrate that MoMKD consistently outperforms state-of-the-art MIL and multimodal KD baselines, delivering strong performance and generalization under histology-only inference. Overall, MoMKD establishes a robust and generalizable knowledge distillation paradigm for computational pathology.

Related papers

• Towards Reliable Stain Transfer: An Iterative Data-Model Co-Optimization Framework Based on Multimodal Expert-Guided Assessment [55.16957949238588]
  We propose DMCoStain, a novel Data-Model Co-optimization framework for Stain transfer.<n>It iteratively co-refines training data and model capability, improving staining accuracy and interpretability in both pathological and structural consistency.<n>To support MEGFS, we construct ImmunoInstruction, the first large-scale IPE instruction-following dataset with 150K VQA samples.
  arXiv  Detail & Related papers  (2026-07-28T07:54:43Z)
• Dementia Etiology Diagnosis via Collaborative Meta Knowledge Enhancement [81.27196033392559]
  We propose a Collaborative Meta Knowledge Enhancement (COME) framework for dementia etiology diagnosis.<n>COME injects multi-center acquisition semantics, source identifiers, and modality indicators as heterogeneity-aware embeddings into a unified Transformer architecture for scale-up training.<n>Our method achieves state-of-the-art in-domain performance with a mean macro-averaged AUC of 85.62% and a 4.29-point gain over the strongest baseline.
  arXiv  Detail & Related papers  (2026-07-24T04:26:47Z)
• Influence-Guided Symbolic Regression: Scientific Discovery via LLM-Driven Equation Search with Granular Feedback [56.69850045068714]
  We introduce textitInfluence-Guided Regression (IGSR), a method that frames equation discovery as an iterative two-step process.<n>We demonstrate IGSRs effectiveness on a diverse suite of benchmarks, including LLM-SRBench, pharmacological PKPD models, an epidemiological simulation, and real-world genomic data.
  arXiv  Detail & Related papers  (2026-05-27T23:48:01Z)
• ConceptM$^3$oE: Concept-Guided Multimodal Mixture of Experts for Interpretable Computational Pathology [33.27476666910628]
  ConceptM$3$oE embeds concept formation directly within interaction-aware mixture-of-experts pathways.<n>The architecture decomposes evidence into modality-specific, redundant, and synergistic experts, which are then projected into structured concept bottlenecks.<n>In data-limited regimes, ConceptM$3$oE improves limited-data performance, increasing macro-F1 from 56.41% to 66.70% at small training sizes.
  arXiv  Detail & Related papers  (2026-05-23T04:55:16Z)
• DOGMA: Weaving Structural Information into Data-centric Single-cell Transcriptomics Analysis [43.565183518761984]
  We propose DOGMA, a data-centric framework designed for the structural reshaping and semantic enhancement of raw data.<n>In complex multi-species and multi-organ benchmarks, DOGMA SOTA performance, exhibiting superior zero-shot robustness and sample efficiency.
  arXiv  Detail & Related papers  (2026-02-02T09:10:09Z)
• Alzheimer's Disease Brain Network Mining [5.586191108738563]
  We introduce Multi view Adaptive Transport Clustering for Heterogeneous Alzheimer's Disease (MATCH-AD)<n>MATCH-AD is a semi supervised framework that integrates deep representation learning, graph-based label propagation, and optimal transport theory.<n>It achieves near-perfect diagnostic accuracy despite ground truth labels for less than one-third of subjects.
  arXiv  Detail & Related papers  (2025-12-19T06:48:54Z)
• Forging a Dynamic Memory: Retrieval-Guided Continual Learning for Generalist Medical Foundation Models [45.285970665585914]
  We propose a comprehensive framework for Continual Learning.<n>We employ a multi-modal, multi-layer RAG system that provides real-time guidance for model fine-tuning.<n>We introduce a dynamic knowledge distillation framework.
  arXiv  Detail & Related papers  (2025-12-15T08:09:40Z)
• Adapting HFMCA to Graph Data: Self-Supervised Learning for Generalizable fMRI Representations [57.054499278843856]
  Functional magnetic resonance imaging (fMRI) analysis faces significant challenges due to limited dataset sizes and domain variability between studies.<n>Traditional self-supervised learning methods inspired by computer vision often rely on positive and negative sample pairs.<n>We propose adapting a recently developed Hierarchical Functional Maximal Correlation Algorithm (HFMCA) to graph-structured fMRI data.
  arXiv  Detail & Related papers  (2025-10-05T12:35:01Z)
• An Interpretable Ensemble Framework for Multi-Omics Dementia Biomarker Discovery Under HDLSS Conditions [0.0]
  We propose a novel ensemble approach combining Graph Attention Networks (GAT), MultiOmics Variational AutoEncoder (MOVE), Elastic-net sparse regression, and Storey's False Discovery Rate (FDR)<n>We evaluate performance using both simulated multi-omics data and the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset.<n>Our method demonstrates superior predictive accuracy, feature selection precision, and biological relevance.
  arXiv  Detail & Related papers  (2025-09-04T15:20:13Z)
• Multi-modal Knowledge Decomposition based Online Distillation for Biomarker Prediction in Breast Cancer Histopathology [9.978792979899376]
  We propose an online distillation approach based on Multi-modal Knowledge Decomposition (MKD) to enhance IHC biomarker prediction.<n>Two teacher and one student models are developed to extract modality-specific and modality-general features.<n>Our approach achieves superior performance in IHC biomarker prediction using uni-modal data.
  arXiv  Detail & Related papers  (2025-08-24T04:56:17Z)
• MIRROR: Multi-Modal Pathological Self-Supervised Representation Learning via Modality Alignment and Retention [57.044719143401664]
  Histopathology and transcriptomics are fundamental modalities in oncology, encapsulating the morphological and molecular aspects of the disease.<n>We present MIRROR, a novel multi-modal representation learning method designed to foster both modality alignment and retention.<n>Extensive evaluations on TCGA cohorts for cancer subtyping and survival analysis highlight MIRROR's superior performance.
  arXiv  Detail & Related papers  (2025-03-01T07:02:30Z)
• Causal Representation Learning from Multimodal Biomedical Observations [57.00712157758845]
  We develop flexible identification conditions for multimodal data and principled methods to facilitate the understanding of biomedical datasets.<n>Key theoretical contribution is the structural sparsity of causal connections between modalities.<n>Results on a real-world human phenotype dataset are consistent with established biomedical research.
  arXiv  Detail & Related papers  (2024-11-10T16:40:27Z)
• Histo-Genomic Knowledge Distillation For Cancer Prognosis From Histopathology Whole Slide Images [7.5123289730388825]
  Genome-informed Hyper-Attention Network (G-HANet) is capable of effectively distilling histo-genomic knowledge during training. Network comprises cross-modal associating branch (CAB) and hyper-attention survival branch (HSB)
  arXiv  Detail & Related papers  (2024-03-15T06:20:09Z)
• Distilling Privileged Multimodal Information for Expression Recognition using Optimal Transport [46.91791643660991]
  Deep learning models for multimodal expression recognition have reached remarkable performance in controlled laboratory environments. These models struggle in the wild because of the unavailability and quality of modalities used for training. In practice, only a subset of the training-time modalities may be available at test time. Learning with privileged information enables models to exploit data from additional modalities that are only available during training.
  arXiv  Detail & Related papers  (2024-01-27T19:44:15Z)
• Brain Imaging-to-Graph Generation using Adversarial Hierarchical Diffusion Models for MCI Causality Analysis [44.45598796591008]
  Brain imaging-to-graph generation (BIGG) framework is proposed to map functional magnetic resonance imaging (fMRI) into effective connectivity for mild cognitive impairment analysis. The hierarchical transformers in the generator are designed to estimate the noise at multiple scales. Evaluations of the ADNI dataset demonstrate the feasibility and efficacy of the proposed model.
  arXiv  Detail & Related papers  (2023-05-18T06:54:56Z)
• Differentiable Agent-based Epidemiology [71.81552021144589]
  We introduce GradABM: a scalable, differentiable design for agent-based modeling that is amenable to gradient-based learning with automatic differentiation. GradABM can quickly simulate million-size populations in few seconds on commodity hardware, integrate with deep neural networks and ingest heterogeneous data sources.
  arXiv  Detail & Related papers  (2022-07-20T07:32:02Z)
• G-MIND: An End-to-End Multimodal Imaging-Genetics Framework for Biomarker Identification and Disease Classification [49.53651166356737]
  We propose a novel deep neural network architecture to integrate imaging and genetics data, as guided by diagnosis, that provides interpretable biomarkers. We have evaluated our model on a population study of schizophrenia that includes two functional MRI (fMRI) paradigms and Single Nucleotide Polymorphism (SNP) data.
  arXiv  Detail & Related papers  (2021-01-27T19:28:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.