seg2med: a bridge from artificial anatomy to multimodal medical images

• URL: http://arxiv.org/abs/2504.09182v2
• Date: Thu, 12 Jun 2025 23:39:43 GMT
• Title: seg2med: a bridge from artificial anatomy to multimodal medical images
• Authors: Zeyu Yang, Zhilin Chen, Yipeng Sun, Anika Strittmatter, Anish Raj, Ahmad Allababidi, Johann S. Rink, Frank G. Zöllner,
• Abstract summary: seg2med is a framework for anatomy-driven multimodal medical image synthesis.<n> anatomical maps are independently derived from three sources: real patient data, XCAT digital phantoms, and synthetic anatomies created by combining organs from multiple patients.<n>The framework achieves SSIM of 0.94 for CT and 0.89 for MR compared to real data, and FSIM of 0.78 for simulated CT.
• Score: 5.92914320764123
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present seg2med, a modular framework for anatomy-driven multimodal medical image synthesis. The system integrates three components to enable high-fidelity, cross-modality generation of CT and MR images based on structured anatomical priors. First, anatomical maps are independently derived from three sources: real patient data, XCAT digital phantoms, and synthetic anatomies created by combining organs from multiple patients. Second, we introduce PhysioSynth, a modality-specific simulator that converts anatomical masks into prior volumes using tissue-dependent parameters (e.g., HU, T1, T2, proton density) and modality-specific signal models. It supports simulation of CT and multiple MR sequences including GRE, SPACE, and VIBE. Third, the synthesized anatomical priors are used to train 2-channel conditional denoising diffusion models, which take the anatomical prior as structural condition alongside the noisy image, enabling generation of high-quality, structurally aligned images. The framework achieves SSIM of 0.94 for CT and 0.89 for MR compared to real data, and FSIM of 0.78 for simulated CT. The generative quality is further supported by a Frechet Inception Distance (FID) of 3.62 for CT synthesis. In modality conversion, seg2med achieves SSIM of 0.91 for MR to CT and 0.77 for CT to MR. Anatomical fidelity evaluation shows synthetic CT achieves mean Dice scores above 0.90 for 11 key abdominal organs, and above 0.80 for 34 of 59 total organs. These results underscore seg2med's utility in cross-modality synthesis, data augmentation, and anatomy-aware medical AI.

Related papers

• Surf2CT: Cascaded 3D Flow Matching Models for Torso 3D CT Synthesis from Skin Surface [13.161605581865357]
  Surf2CT is a framework that synthesizes full 3D computed tomography (CT) volumes of the human torso from external surface scans and demographic data.<n>We trained our model on a combined dataset of 3,198 torso CT scans sourced from Massachusetts General Hospital (MGH) and the AutoPET challenge.
  arXiv  Detail & Related papers  (2025-05-28T16:01:36Z)
• JSover: Joint Spectrum Estimation and Multi-Material Decomposition from Single-Energy CT Projections [45.14515691206885]
  Multi-material decomposition (MMD) enables quantitative reconstruction of tissue compositions in the human body.<n>Traditional MMD typically requires spectral CT scanners and pre-measured X-ray energy spectra, significantly limiting clinical applicability.<n>This paper proposes JSover, a fundamentally reformulated one-step SEMMD framework that jointly reconstructs multi-material compositions and estimates the energy spectrum directly from SECT projections.
  arXiv  Detail & Related papers  (2025-05-12T23:32:21Z)
• Revolutionizing Brain Tumor Imaging: Generating Synthetic 3D FA Maps from T1-Weighted MRI using CycleGAN Models [18.167577989282247]
  We propose a CycleGAN based approach for generating FA maps directly from T1-weighted MRI scans.<n>Our model, trained on unpaired data, produces high fidelity maps, demonstrating particularly robust performance in tumour regions.
  arXiv  Detail & Related papers  (2025-05-06T16:05:22Z)
• SYN-LUNGS: Towards Simulating Lung Nodules with Anatomy-Informed Digital Twins for AI Training [1.4820790252910163]
  Generative models for lung cancer screening are limited by data scarcity, impacting generalizability and clinical applicability.<n>We introduce SYN-LUNGS, a framework for generating high-quality 3D CT images with detailed annotations.<n>The dataset includes 3,072 nodule images from 1,044 simulated CT scans, with 512 lesions and 174 digital twins.
  arXiv  Detail & Related papers  (2025-02-28T16:02:37Z)
• MRI Breast tissue segmentation using nnU-Net for biomechanical modeling [0.07499722271664144]
  Integrating 2D mammography with 3D magnetic resonance imaging (MRI) is crucial for improving breast cancer diagnosis and treatment planning.<n>This paper addresses these challenges by enhancing biomechanical breast models in two main aspects.<n>We performed a detailed six-class segmentation of breast MRI data using the nnU-Net architecture.<n>The overall foreground segmentation reached a mean Dice Coefficient of 0.83 through an ensemble of 2D and 3D U-Net configurations.
  arXiv  Detail & Related papers  (2024-11-27T22:15:04Z)
• Multi-Layer Feature Fusion with Cross-Channel Attention-Based U-Net for Kidney Tumor Segmentation [0.0]
  U-Net based deep learning techniques are emerging as a promising approach for automated medical image segmentation. We present an improved U-Net based model for end-to-end automated semantic segmentation of CT scan images to identify renal tumors.
  arXiv  Detail & Related papers  (2024-10-20T19:02:41Z)
• Improved 3D Whole Heart Geometry from Sparse CMR Slices [3.701571763780745]
  Cardiac magnetic resonance (CMR) imaging and computed tomography (CT) are two common non-invasive imaging methods for assessing patients with cardiovascular disease. CMR typically acquires multiple sparse 2D slices, with unavoidable respiratory motion artefacts between slices, whereas CT acquires isotropic dense data but uses ionising radiation. We explore the combination of Slice Shifting Algorithm (SSA), Spatial Transformer Network (STN), and Label Transformer Network (LTN) to: 1) correct respiratory motion between segmented slices, and 2) transform sparse segmentation data into dense segmentation.
  arXiv  Detail & Related papers  (2024-08-14T13:03:48Z)
• Deep learning-based brain segmentation model performance validation with clinical radiotherapy CT [0.0]
  This study validates the SynthSeg robust brain segmentation model on computed tomography (CT) Brain segmentations from CT and MRI were obtained with SynthSeg model, a component of the Freesurfer imaging suite. CT performance is lower than MRI based on the integrated QC scores, but low-quality segmentations can be excluded with QC-based thresholding.
  arXiv  Detail & Related papers  (2024-06-25T09:56:30Z)
• TotalSegmentator MRI: Robust Sequence-independent Segmentation of Multiple Anatomic Structures in MRI [59.86827659781022]
  A nnU-Net model (TotalSegmentator) was trained on MRI and segment 80atomic structures.<n>Dice scores were calculated between the predicted segmentations and expert reference standard segmentations to evaluate model performance.<n>Open-source, easy-to-use model allows for automatic, robust segmentation of 80 structures.
  arXiv  Detail & Related papers  (2024-05-29T20:15:54Z)
• Analysis of the BraTS 2023 Intracranial Meningioma Segmentation Challenge [44.76736949127792]
  We describe the design and results from the BraTS 2023 Intracranial Meningioma Challenge.<n>The BraTS Meningioma Challenge differed from prior BraTS Glioma challenges in that it focused on meningiomas.<n>The top ranked team had a lesion-wise median dice similarity coefficient (DSC) of 0.976, 0.976, and 0.964 for enhancing tumor, tumor core, and whole tumor.
  arXiv  Detail & Related papers  (2024-05-16T03:23:57Z)
• MRSegmentator: Multi-Modality Segmentation of 40 Classes in MRI and CT [29.48170108608303]
  The model was trained on 1,200 manually annotated 3D axial MRI scans from the UK Biobank, 221 in-house MRI scans, and 1228 CT scans. It demonstrated high accuracy for well-defined organs (lungs: DSC 0.96, heart: DSC 0.94) and organs with anatomic variability (liver: DSC 0.96, kidneys: DSC 0.95) It generalized well to CT, achieving DSC mean of 0.84 $pm$ 0.11 on AMOS CT data.
  arXiv  Detail & Related papers  (2024-05-10T13:15:42Z)
• SDR-Former: A Siamese Dual-Resolution Transformer for Liver Lesion Classification Using 3D Multi-Phase Imaging [59.78761085714715]
  This study proposes a novel Siamese Dual-Resolution Transformer (SDR-Former) framework for liver lesion classification. The proposed framework has been validated through comprehensive experiments on two clinical datasets. To support the scientific community, we are releasing our extensive multi-phase MR dataset for liver lesion analysis to the public.
  arXiv  Detail & Related papers  (2024-02-27T06:32:56Z)
• Minimally Interactive Segmentation of Soft-Tissue Tumors on CT and MRI using Deep Learning [0.0]
  We develop a minimally interactive deep learning-based segmentation method for soft-tissue tumors (STTs) on CT and MRI. The method requires the user to click six points near the tumor's extreme boundaries to serve as input for a Convolutional Neural Network.
  arXiv  Detail & Related papers  (2024-02-12T16:15:28Z)
• A Two-Stage Generative Model with CycleGAN and Joint Diffusion for MRI-based Brain Tumor Detection [41.454028276986946]
  We propose a novel framework Two-Stage Generative Model (TSGM) to improve brain tumor detection and segmentation. CycleGAN is trained on unpaired data to generate abnormal images from healthy images as data prior. VE-JP is implemented to reconstruct healthy images using synthetic paired abnormal images as a guide.
  arXiv  Detail & Related papers  (2023-11-06T12:58:26Z)
• Accurate Fine-Grained Segmentation of Human Anatomy in Radiographs via Volumetric Pseudo-Labeling [66.75096111651062]
  We created a large-scale dataset of 10,021 thoracic CTs with 157 labels. We applied an ensemble of 3D anatomy segmentation models to extract anatomical pseudo-labels. Our resulting segmentation models demonstrated remarkable performance on CXR.
  arXiv  Detail & Related papers  (2023-06-06T18:01:08Z)
• Evaluation of Synthetically Generated CT for use in Transcranial Focused Ultrasound Procedures [5.921808547303054]
  Transcranial focused ultrasound (tFUS) is a therapeutic ultrasound method that focuses sound through the skull to a small region noninvasively and often under MRI guidance. CT imaging is used to estimate the acoustic properties that vary between individual skulls to enable effective focusing during tFUS procedures. Here, we synthesized CT images from routinely acquired T1-weighted MRI by using a 3D patch-based conditional generative adversarial network (cGAN) We compared the performance of sCT to real CT (rCT) images for tFUS planning using Kranion and simulations using the acoustic toolbox,
  arXiv  Detail & Related papers  (2022-10-26T15:15:24Z)
• TotalSegmentator: robust segmentation of 104 anatomical structures in CT images [48.50994220135258]
  We present a deep learning segmentation model for body CT images. The model can segment 104 anatomical structures relevant for use cases such as organ volumetry, disease characterization, and surgical or radiotherapy planning.
  arXiv  Detail & Related papers  (2022-08-11T15:16:40Z)
• Negligible effect of brain MRI data preprocessing for tumor segmentation [36.89606202543839]
  We conduct experiments on three publicly available datasets and evaluate the effect of different preprocessing steps in deep neural networks. Our results demonstrate that most popular standardization steps add no value to the network performance. We suggest that image intensity normalization approaches do not contribute to model accuracy because of the reduction of signal variance with image standardization.
  arXiv  Detail & Related papers  (2022-04-11T17:29:36Z)
• iPhantom: a framework for automated creation of individualized computational phantoms and its application to CT organ dosimetry [58.943644554192936]
  This study aims to develop and validate a novel framework, iPhantom, for automated creation of patient-specific phantoms or digital-twins. The framework is applied to assess radiation dose to radiosensitive organs in CT imaging of individual patients. iPhantom precisely predicted all organ locations with good accuracy of Dice Similarity Coefficients (DSC) >0.6 for anchor organs and DSC of 0.3-0.9 for all other organs.
  arXiv  Detail & Related papers  (2020-08-20T01:50:49Z)
• Confidence-guided Lesion Mask-based Simultaneous Synthesis of Anatomic and Molecular MR Images in Patients with Post-treatment Malignant Gliomas [65.64363834322333]
  Confidence Guided SAMR (CG-SAMR) synthesizes data from lesion information to multi-modal anatomic sequences. module guides the synthesis based on confidence measure about the intermediate results. experiments on real clinical data demonstrate that the proposed model can perform better than the state-of-theart synthesis methods.
  arXiv  Detail & Related papers  (2020-08-06T20:20:22Z)
• Lesion Mask-based Simultaneous Synthesis of Anatomic and MolecularMR Images using a GAN [59.60954255038335]
  The proposed framework consists of a stretch-out up-sampling module, a brain atlas encoder, a segmentation consistency module, and multi-scale label-wise discriminators. Experiments on real clinical data demonstrate that the proposed model can perform significantly better than the state-of-the-art synthesis methods.
  arXiv  Detail & Related papers  (2020-06-26T02:50:09Z)
• Co-Heterogeneous and Adaptive Segmentation from Multi-Source and Multi-Phase CT Imaging Data: A Study on Pathological Liver and Lesion Segmentation [48.504790189796836]
  We present a novel segmentation strategy, co-heterogenous and adaptive segmentation (CHASe) We propose a versatile framework that fuses appearance based semi-supervision, mask based adversarial domain adaptation, and pseudo-labeling. CHASe can further improve pathological liver mask Dice-Sorensen coefficients by ranges of $4.2% sim 9.4%$.
  arXiv  Detail & Related papers  (2020-05-27T06:58:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.