Cyclic 2.5D Perceptual Loss for Cross-Modal 3D Medical Image Synthesis: T1w MRI to Tau PET

• URL: http://arxiv.org/abs/2406.12632v2
• Date: Thu, 15 May 2025 05:17:41 GMT
• Title: Cyclic 2.5D Perceptual Loss for Cross-Modal 3D Medical Image Synthesis: T1w MRI to Tau PET
• Authors: Junho Moon, Symac Kim, Haejun Chung, Ikbeom Jang,
• Abstract summary: We propose a cyclic 2.5D perceptual loss that computes the 2D average perceptual loss for each of the axial, coronal, and sagittal planes over epochs.<n>We process tau PET images using by-manufacturer standardization to enhance the preservation of high-SUVR regions.
• Score: 0.04924932828166548
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: There is a demand for medical image synthesis or translation to generate synthetic images of missing modalities from available data. This need stems from challenges such as restricted access to high-cost imaging devices, government regulations, or failure to follow up with patients or study participants. In medical imaging, preserving high-level semantic features is often more critical than achieving pixel-level accuracy. Perceptual loss functions are widely employed to train medical image synthesis or translation models, as they quantify differences in high-level image features using a pre-trained feature extraction network. While 3D and 2.5D perceptual losses are used in 3D medical image synthesis, they face challenges, such as the lack of pre-trained 3D models or difficulties in balancing loss reduction across different planes. In this work, we focus on synthesizing 3D tau PET images from 3D T1-weighted MR images. We propose a cyclic 2.5D perceptual loss that sequentially computes the 2D average perceptual loss for each of the axial, coronal, and sagittal planes over epochs, with the cycle duration gradually decreasing. Additionally, we process tau PET images using by-manufacturer standardization to enhance the preservation of high-SUVR regions indicative of tau pathology and mitigate SUVR variability caused by inter-manufacturer differences. We combine the proposed loss with SSIM and MSE losses and demonstrate its effectiveness in improving both quantitative and qualitative performance across various generative models, including U-Net, UNETR, SwinUNETR, CycleGAN, and Pix2Pix.

Related papers

• End-to-end Triple-domain PET Enhancement: A Hybrid Denoising-and-reconstruction Framework for Reconstructing Standard-dose PET Images from Low-dose PET Sinograms [43.13562515963306]
  We propose an end-to-end TriPle-domain LPET EnhancemenT (TriPLET) framework to reconstruct standard-dose PET images from low-dose PET sinograms. Our proposed TriPLET can reconstruct SPET images with the highest similarity and signal-to-noise ratio to real data, compared with state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-12-04T14:47:27Z)
• Multi-resolution Guided 3D GANs for Medical Image Translation [6.299981733052469]
  We introduce a multi-resolution guided Generative Adrial Network (GAN)-based framework for 3D medical image translation.<n>Our framework uses a 3D multi-resolution Dense-Attention UNet (3D-mDAUNet) as the generator and a 3D multi-resolution UNet as the discriminator.<n>Our approach yields promising results in volumetric image quality assessment (IQA) across a variety of imaging modalities, body regions, and age groups, demonstrating its robustness.
  arXiv  Detail & Related papers  (2024-11-30T20:11:55Z)
• Cycle-Constrained Adversarial Denoising Convolutional Network for PET Image Denoising: Multi-Dimensional Validation on Large Datasets with Reader Study and Real Low-Dose Data [9.160782425067712]
  We propose a Cycle-versa Adrial Denoising Convolutional Network (Cycle-DCN) to reconstruct full-dose-quality images from low-dose scans. Experiments were conducted on a large dataset consisting of raw PET brain data from 1,224 patients. Cycle-DCN significantly improves average Peak Signal-to-Noise Ratio (PSNR), SSIM, and Normalized Root Mean Square Error (NRMSE) across three dose levels.
  arXiv  Detail & Related papers  (2024-10-31T04:34:28Z)
• Diff3Dformer: Leveraging Slice Sequence Diffusion for Enhanced 3D CT Classification with Transformer Networks [5.806035963947936]
  We propose a Diffusion-based 3D Vision Transformer (Diff3Dformer) to aggregate repetitive information within 3D CT scans. Our method exhibits improved performance on two different scales of small datasets of 3D lung CT scans.
  arXiv  Detail & Related papers  (2024-06-24T23:23:18Z)
• 2.5D Multi-view Averaging Diffusion Model for 3D Medical Image Translation: Application to Low-count PET Reconstruction with CT-less Attenuation Correction [17.897681480967087]
  Positron Emission Tomography (PET) is an important clinical imaging tool but inevitably introduces radiation hazards to patients and healthcare providers. It is desirable to develop 3D methods to translate the non-attenuation-corrected low-dose PET into attenuation-corrected standard-dose PET. Recent diffusion models have emerged as a new state-of-the-art deep learning method for image-to-image translation, better than traditional CNN-based methods. We developed a novel 2.5D Multi-view Averaging Diffusion Model (MADM) for 3D image-to-image translation with application on NAC
  arXiv  Detail & Related papers  (2024-06-12T16:22:41Z)
• 3D Vessel Reconstruction from Sparse-View Dynamic DSA Images via Vessel Probability Guided Attenuation Learning [79.60829508459753]
  Current commercial Digital Subtraction Angiography (DSA) systems typically demand hundreds of scanning views to perform reconstruction. The dynamic blood flow and insufficient input of sparse-view DSA images present significant challenges to the 3D vessel reconstruction task. We propose to use a time-agnostic vessel probability field to solve this problem effectively.
  arXiv  Detail & Related papers  (2024-05-17T11:23:33Z)
• Functional Imaging Constrained Diffusion for Brain PET Synthesis from Structural MRI [5.190302448685122]
  We propose a framework for 3D brain PET image synthesis with paired structural MRI as input condition, through a new constrained diffusion model (CDM) The FICD introduces noise to PET and then progressively removes it with CDM, ensuring high output fidelity throughout a stable training phase. The CDM learns to predict denoised PET with a functional imaging constraint introduced to ensure voxel-wise alignment between each denoised PET and its ground truth.
  arXiv  Detail & Related papers  (2024-05-03T22:33:46Z)
• Three-Dimensional Amyloid-Beta PET Synthesis from Structural MRI with Conditional Generative Adversarial Networks [45.426889188365685]
  Alzheimer's Disease hallmarks include amyloid-beta deposits and brain atrophy. PET is expensive, invasive and exposes patients to ionizing radiation. MRI is cheaper, non-invasive, and free from ionizing radiation but limited to measuring brain atrophy.
  arXiv  Detail & Related papers  (2024-05-03T14:10:29Z)
• CycleINR: Cycle Implicit Neural Representation for Arbitrary-Scale Volumetric Super-Resolution of Medical Data [19.085329423308938]
  CycleINR is a novel enhanced Implicit Neural Representation model for 3D medical data super-resolution. We introduce a new metric, Slice-wise Noise Level Inconsistency (SNLI), to quantitatively assess inter-slice noise level inconsistency.
  arXiv  Detail & Related papers  (2024-04-07T08:48:01Z)
• Generative Enhancement for 3D Medical Images [74.17066529847546]
  We propose GEM-3D, a novel generative approach to the synthesis of 3D medical images. Our method begins with a 2D slice, noted as the informed slice to serve the patient prior, and propagates the generation process using a 3D segmentation mask. By decomposing the 3D medical images into masks and patient prior information, GEM-3D offers a flexible yet effective solution for generating versatile 3D images.
  arXiv  Detail & Related papers  (2024-03-19T15:57:04Z)
• Rotational Augmented Noise2Inverse for Low-dose Computed Tomography Reconstruction [83.73429628413773]
  Supervised deep learning methods have shown the ability to remove noise in images but require accurate ground truth. We propose a novel self-supervised framework for LDCT, in which ground truth is not required for training the convolutional neural network (CNN) Numerical and experimental results show that the reconstruction accuracy of N2I with sparse views is degrading while the proposed rotational augmented Noise2Inverse (RAN2I) method keeps better image quality over a different range of sampling angles.
  arXiv  Detail & Related papers  (2023-12-19T22:40:51Z)
• StableDreamer: Taming Noisy Score Distillation Sampling for Text-to-3D [88.66678730537777]
  We present StableDreamer, a methodology incorporating three advances. First, we formalize the equivalence of the SDS generative prior and a simple supervised L2 reconstruction loss. Second, our analysis shows that while image-space diffusion contributes to geometric precision, latent-space diffusion is crucial for vivid color rendition.
  arXiv  Detail & Related papers  (2023-12-02T02:27:58Z)
• PET Synthesis via Self-supervised Adaptive Residual Estimation Generative Adversarial Network [14.381830012670969]
  Recent methods to generate high-quality PET images from low-dose counterparts have been reported to be state-of-the-art for low-to-high image recovery methods. To address these issues, we developed a self-supervised adaptive residual estimation generative adversarial network (SS-AEGAN) SS-AEGAN consistently outperformed the state-of-the-art synthesis methods with various dose reduction factors.
  arXiv  Detail & Related papers  (2023-10-24T06:43:56Z)
• Contrastive Diffusion Model with Auxiliary Guidance for Coarse-to-Fine PET Reconstruction [62.29541106695824]
  This paper presents a coarse-to-fine PET reconstruction framework that consists of a coarse prediction module (CPM) and an iterative refinement module (IRM) By delegating most of the computational overhead to the CPM, the overall sampling speed of our method can be significantly improved. Two additional strategies, i.e., an auxiliary guidance strategy and a contrastive diffusion strategy, are proposed and integrated into the reconstruction process.
  arXiv  Detail & Related papers  (2023-08-20T04:10:36Z)
• TriDo-Former: A Triple-Domain Transformer for Direct PET Reconstruction from Low-Dose Sinograms [45.24575167909925]
  TriDoFormer is a transformer-based model that unites triple domains of sinogram, image, and frequency for direct reconstruction. It outperforms state-of-the-art methods qualitatively and quantitatively. GFP serves as a learnable frequency filter that adjusts the frequency components in the frequency domain, enforcing the network to restore high-frequency details.
  arXiv  Detail & Related papers  (2023-08-10T06:20:00Z)
• On Sensitivity and Robustness of Normalization Schemes to Input Distribution Shifts in Automatic MR Image Diagnosis [58.634791552376235]
  Deep Learning (DL) models have achieved state-of-the-art performance in diagnosing multiple diseases using reconstructed images as input. DL models are sensitive to varying artifacts as it leads to changes in the input data distribution between the training and testing phases. We propose to use other normalization techniques, such as Group Normalization and Layer Normalization, to inject robustness into model performance against varying image artifacts.
  arXiv  Detail & Related papers  (2023-06-23T03:09:03Z)
• CG-3DSRGAN: A classification guided 3D generative adversarial network for image quality recovery from low-dose PET images [10.994223928445589]
  High radioactivity caused by the injected tracer dose is a major concern in PET imaging. Reducing the dose leads to inadequate image quality for diagnostic practice. CNNs-based methods have been developed for high quality PET synthesis from its low-dose counterparts.
  arXiv  Detail & Related papers  (2023-04-03T05:39:02Z)
• Synthetic PET via Domain Translation of 3D MRI [1.0052333944678682]
  We use a dataset of 56 $18$F-FDG-PET/MRI exams to train a 3D residual UNet to predict physiologic PET uptake from whole-body T1-weighted MRI. The predicted PET images are forward projected to produce synthetic PET time-of-flight sinograms that can be used with vendor-provided PET reconstruction algorithms.
  arXiv  Detail & Related papers  (2022-06-11T21:32:40Z)
• CyTran: A Cycle-Consistent Transformer with Multi-Level Consistency for Non-Contrast to Contrast CT Translation [56.622832383316215]
  We propose a novel approach to translate unpaired contrast computed tomography (CT) scans to non-contrast CT scans. Our approach is based on cycle-consistent generative adversarial convolutional transformers, for short, CyTran. Our empirical results show that CyTran outperforms all competing methods.
  arXiv  Detail & Related papers  (2021-10-12T23:25:03Z)
• Revisiting 3D Context Modeling with Supervised Pre-training for Universal Lesion Detection in CT Slices [48.85784310158493]
  We propose a Modified Pseudo-3D Feature Pyramid Network (MP3D FPN) to efficiently extract 3D context enhanced 2D features for universal lesion detection in CT slices. With the novel pre-training method, the proposed MP3D FPN achieves state-of-the-art detection performance on the DeepLesion dataset. The proposed 3D pre-trained weights can potentially be used to boost the performance of other 3D medical image analysis tasks.
  arXiv  Detail & Related papers  (2020-12-16T07:11:16Z)
• Tattoo tomography: Freehand 3D photoacoustic image reconstruction with an optical pattern [49.240017254888336]
  Photoacoustic tomography (PAT) is a novel imaging technique that can resolve both morphological and functional tissue properties. A current drawback is the limited field-of-view provided by the conventionally applied 2D probes. We present a novel approach to 3D reconstruction of PAT data that does not require an external tracking system.
  arXiv  Detail & Related papers  (2020-11-10T09:27:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.