A Semantically Enhanced Generative Foundation Model Improves Pathological Image Synthesis

• URL: http://arxiv.org/abs/2512.13164v2
• Date: Tue, 16 Dec 2025 03:25:24 GMT
• Title: A Semantically Enhanced Generative Foundation Model Improves Pathological Image Synthesis
• Authors: Xianchao Guan, Zhiyuan Fan, Yifeng Wang, Fuqiang Chen, Yanjiang Zhou, Zengyang Che, Hongxue Meng, Xin Li, Yaowei Wang, Hongpeng Wang, Min Zhang, Heng Tao Shen, Zheng Zhang, Yongbing Zhang,
• Abstract summary: We introduce a Correlation-Regulated Alignment Framework for Tissue Synthesis (CRAFTS) for pathology-specific text-to-image synthesis.<n>CRAFTS incorporates a novel alignment mechanism that suppresses semantic drift to ensure biological accuracy.<n>This model generates diverse pathological images spanning 30 cancer types, with quality rigorously validated by objective metrics and pathologist evaluations.
• Score: 82.01597026329158
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The development of clinical-grade artificial intelligence in pathology is limited by the scarcity of diverse, high-quality annotated datasets. Generative models offer a potential solution but suffer from semantic instability and morphological hallucinations that compromise diagnostic reliability. To address this challenge, we introduce a Correlation-Regulated Alignment Framework for Tissue Synthesis (CRAFTS), the first generative foundation model for pathology-specific text-to-image synthesis. By leveraging a dual-stage training strategy on approximately 2.8 million image-caption pairs, CRAFTS incorporates a novel alignment mechanism that suppresses semantic drift to ensure biological accuracy. This model generates diverse pathological images spanning 30 cancer types, with quality rigorously validated by objective metrics and pathologist evaluations. Furthermore, CRAFTS-augmented datasets enhance the performance across various clinical tasks, including classification, cross-modal retrieval, self-supervised learning, and visual question answering. In addition, coupling CRAFTS with ControlNet enables precise control over tissue architecture from inputs such as nuclear segmentation masks and fluorescence images. By overcoming the critical barriers of data scarcity and privacy concerns, CRAFTS provides a limitless source of diverse, annotated histology data, effectively unlocking the creation of robust diagnostic tools for rare and complex cancer phenotypes.

Related papers

• A WDLoRA-Based Multimodal Generative Framework for Clinically Guided Corneal Confocal Microscopy Image Synthesis in Diabetic Neuropathy [8.701084151107652]
  Corneal Confocal Microscopy is a sensitive tool for assessing small-fiber damage in Diabetic Peripheral Neuropathy (DPN)<n>Development of robust, automated deep learning-based diagnostic models is limited by scarce labelled data and fine-grained variability in corneal nerve morphology.<n>We propose a Weight-Decomposed Low-Rank Adaptation (WDLoRA)-based multimodal generative framework for clinically guided CCM image synthesis.
  arXiv  Detail & Related papers  (2026-02-14T09:32:44Z)
• Causal Disentanglement for Robust Long-tail Medical Image Generation [80.15257897500578]
  We propose a novel medical image generation framework, which generates independent pathological and structural features.<n>We leverage a diffusion model guided by pathological findings to model pathological features, enabling the generation of diverse counterfactual images.
  arXiv  Detail & Related papers  (2025-04-20T01:54:18Z)
• Towards Accurate and Interpretable Neuroblastoma Diagnosis via Contrastive Multi-scale Pathological Image Analysis [16.268045905735818]
  We propose CMSwinKAN, a contrastive-learning-based multi-scale feature fusion model tailored for pathological image classification.<n>By fusing multi-scale features and leveraging contrastive learning strategies, CMSwinKAN mimics clinicians' comprehensive approach.<n>Results demonstrate that CMSwinKAN performs better than existing state-of-the-art pathology-specific models pre-trained on large datasets.
  arXiv  Detail & Related papers  (2025-04-18T15:39:46Z)
• FundusGAN: A Hierarchical Feature-Aware Generative Framework for High-Fidelity Fundus Image Generation [35.46876389599076]
  FundusGAN is a novel hierarchical feature-aware generative framework specifically designed for high-fidelity fundus image synthesis.<n>We show that FundusGAN consistently outperforms state-of-the-art methods across multiple metrics.
  arXiv  Detail & Related papers  (2025-03-22T18:08:07Z)
• HistoSPACE: Histology-Inspired Spatial Transcriptome Prediction And Characterization Engine [0.0]
  HistoSPACE model explore the diversity of histological images available with ST data to extract molecular insights from tissue image. Model demonstrates significant efficiency compared to contemporary algorithms, revealing a correlation of 0.56 in leave-one-out cross-validation.
  arXiv  Detail & Related papers  (2024-08-07T07:12:52Z)
• PathLDM: Text conditioned Latent Diffusion Model for Histopathology [62.970593674481414]
  We introduce PathLDM, the first text-conditioned Latent Diffusion Model tailored for generating high-quality histopathology images. Our approach fuses image and textual data to enhance the generation process. We achieved a SoTA FID score of 7.64 for text-to-image generation on the TCGA-BRCA dataset, significantly outperforming the closest text-conditioned competitor with FID 30.1.
  arXiv  Detail & Related papers  (2023-09-01T22:08:32Z)
• Feature Representation Learning for Robust Retinal Disease Detection from Optical Coherence Tomography Images [0.0]
  Ophthalmic images may contain identical-looking pathologies that can cause failure in automated techniques to distinguish different retinal degenerative diseases. In this work, we propose a robust disease detection architecture with three learning heads. Our experimental results on two publicly available OCT datasets illustrate that the proposed model outperforms existing state-of-the-art models in terms of accuracy, interpretability, and robustness for out-of-distribution retinal disease detection.
  arXiv  Detail & Related papers  (2022-06-24T07:59:36Z)
• Harmonizing Pathological and Normal Pixels for Pseudo-healthy Synthesis [68.5287824124996]
  We present a new type of discriminator, the segmentor, to accurately locate the lesions and improve the visual quality of pseudo-healthy images. We apply the generated images into medical image enhancement and utilize the enhanced results to cope with the low contrast problem. Comprehensive experiments on the T2 modality of BraTS demonstrate that the proposed method substantially outperforms the state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-03-29T08:41:17Z)
• Data-driven generation of plausible tissue geometries for realistic photoacoustic image synthesis [53.65837038435433]
  Photoacoustic tomography (PAT) has the potential to recover morphological and functional tissue properties. We propose a novel approach to PAT data simulation, which we refer to as "learning to simulate" We leverage the concept of Generative Adversarial Networks (GANs) trained on semantically annotated medical imaging data to generate plausible tissue geometries.
  arXiv  Detail & Related papers  (2021-03-29T11:30:18Z)
• G-MIND: An End-to-End Multimodal Imaging-Genetics Framework for Biomarker Identification and Disease Classification [49.53651166356737]
  We propose a novel deep neural network architecture to integrate imaging and genetics data, as guided by diagnosis, that provides interpretable biomarkers. We have evaluated our model on a population study of schizophrenia that includes two functional MRI (fMRI) paradigms and Single Nucleotide Polymorphism (SNP) data.
  arXiv  Detail & Related papers  (2021-01-27T19:28:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.