Diffusion Transformer Meets Random Masks: An Advanced PET Reconstruction Framework

• URL: http://arxiv.org/abs/2503.08339v1
• Date: Tue, 11 Mar 2025 11:50:50 GMT
• Title: Diffusion Transformer Meets Random Masks: An Advanced PET Reconstruction Framework
• Authors: Bin Huang, Binzhong He, Yanhan Chen, Zhili Liu, Xinyue Wang, Binxuan Li, Qiegen Liu,
• Abstract summary: In this study, we pro-pose an advanced PET reconstruction framework called Diffusion tRansformer mEets rAndom Masks (DREAM)<n>To the best of our knowledge, this is the first work to integrate mask mechanisms into both the sinogram domain and the latent space.<n> Experimental results demonstrate that DREAM not only improves the overall quality of reconstructed PET images but also preserves critical clinical details.
• Score: 11.726122474828065
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deep learning has significantly advanced PET image re-construction, achieving remarkable improvements in image quality through direct training on sinogram or image data. Traditional methods often utilize masks for inpainting tasks, but their incorporation into PET reconstruction frameworks introduces transformative potential. In this study, we pro-pose an advanced PET reconstruction framework called Diffusion tRansformer mEets rAndom Masks (DREAM). To the best of our knowledge, this is the first work to integrate mask mechanisms into both the sinogram domain and the latent space, pioneering their role in PET reconstruction and demonstrating their ability to enhance reconstruction fidelity and efficiency. The framework employs a high-dimensional stacking approach, transforming masked data from two to three dimensions to expand the solution space and enable the model to capture richer spatial rela-tionships. Additionally, a mask-driven latent space is de-signed to accelerate the diffusion process by leveraging sinogram-driven and mask-driven compact priors, which reduce computational complexity while preserving essen-tial data characteristics. A hierarchical masking strategy is also introduced, guiding the model from focusing on fi-ne-grained local details in the early stages to capturing broader global patterns over time. This progressive ap-proach ensures a balance between detailed feature preservation and comprehensive context understanding. Experimental results demonstrate that DREAM not only improves the overall quality of reconstructed PET images but also preserves critical clinical details, highlighting its potential to advance PET imaging technology. By inte-grating compact priors and hierarchical masking, DREAM offers a promising and efficient avenue for future research and application in PET imaging. The open-source code is available at: https://github.com/yqx7150/DREAM.

Related papers

• Multi-Subject Image Synthesis as a Generative Prior for Single-Subject PET Image Reconstruction [40.34650079545031]
  We propose a novel method for synthesising diverse and realistic pseudo-PET images with improved signal-to-noise ratio.<n>We show how our pseudo-PET images may be exploited as a generative prior for single-subject PET image reconstruction.
  arXiv  Detail & Related papers  (2024-12-05T16:40:33Z)
• UGMAE: A Unified Framework for Graph Masked Autoencoders [67.75493040186859]
  We propose UGMAE, a unified framework for graph masked autoencoders. We first develop an adaptive feature mask generator to account for the unique significance of nodes. We then design a ranking-based structure reconstruction objective joint with feature reconstruction to capture holistic graph information.
  arXiv  Detail & Related papers  (2024-02-12T19:39:26Z)
• Contrastive Diffusion Model with Auxiliary Guidance for Coarse-to-Fine PET Reconstruction [62.29541106695824]
  This paper presents a coarse-to-fine PET reconstruction framework that consists of a coarse prediction module (CPM) and an iterative refinement module (IRM) By delegating most of the computational overhead to the CPM, the overall sampling speed of our method can be significantly improved. Two additional strategies, i.e., an auxiliary guidance strategy and a contrastive diffusion strategy, are proposed and integrated into the reconstruction process.
  arXiv  Detail & Related papers  (2023-08-20T04:10:36Z)
• TriDo-Former: A Triple-Domain Transformer for Direct PET Reconstruction from Low-Dose Sinograms [45.24575167909925]
  TriDoFormer is a transformer-based model that unites triple domains of sinogram, image, and frequency for direct reconstruction. It outperforms state-of-the-art methods qualitatively and quantitatively. GFP serves as a learnable frequency filter that adjusts the frequency components in the frequency domain, enforcing the network to restore high-frequency details.
  arXiv  Detail & Related papers  (2023-08-10T06:20:00Z)
• Disruptive Autoencoders: Leveraging Low-level features for 3D Medical Image Pre-training [51.16994853817024]
  This work focuses on designing an effective pre-training framework for 3D radiology images. We introduce Disruptive Autoencoders, a pre-training framework that attempts to reconstruct the original image from disruptions created by a combination of local masking and low-level perturbations. The proposed pre-training framework is tested across multiple downstream tasks and achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-07-31T17:59:42Z)
• STPDnet: Spatial-temporal convolutional primal dual network for dynamic PET image reconstruction [16.47493157003075]
  We propose a spatial-temporal convolutional primal dual network (STPDnet) for dynamic PET image reconstruction. The physical projection of PET is embedded in the iterative learning process of the network. Experiments have shown that the proposed method can achieve substantial noise in both temporal and spatial domains.
  arXiv  Detail & Related papers  (2023-03-08T15:43:15Z)
• Fully 3D Implementation of the End-to-end Deep Image Prior-based PET Image Reconstruction Using Block Iterative Algorithm [0.0]
  Deep image prior (DIP) has attracted attention owing to its unsupervised positron emission tomography (PET) image reconstruction. We present the first attempt to implement an end-to-end DIP-based fully 3D PET image reconstruction method.
  arXiv  Detail & Related papers  (2022-12-22T16:25:58Z)
• MAT: Mask-Aware Transformer for Large Hole Image Inpainting [79.67039090195527]
  We present a novel model for large hole inpainting, which unifies the merits of transformers and convolutions. Experiments demonstrate the state-of-the-art performance of the new model on multiple benchmark datasets.
  arXiv  Detail & Related papers  (2022-03-29T06:36:17Z)
• Direct PET Image Reconstruction Incorporating Deep Image Prior and a Forward Projection Model [0.0]
  Convolutional neural networks (CNNs) have recently achieved remarkable performance in positron emission tomography (PET) image reconstruction. We propose an unsupervised direct PET image reconstruction method that incorporates a deep image prior framework. Our proposed method incorporates a forward projection model with a loss function to achieve unsupervised direct PET image reconstruction from sinograms.
  arXiv  Detail & Related papers  (2021-09-02T08:07:58Z)
• TransRPPG: Remote Photoplethysmography Transformer for 3D Mask Face Presentation Attack Detection [53.98866801690342]
  3D mask face presentation attack detection (PAD) plays a vital role in securing face recognition systems from 3D mask attacks. We propose a pure r transformer (TransR) framework for learning live intrinsicness representation efficiently. Our TransR is lightweight and efficient (with only 547K parameters and 763MOPs) which is promising for mobile-level applications.
  arXiv  Detail & Related papers  (2021-04-15T12:33:13Z)
• TFill: Image Completion via a Transformer-Based Architecture [69.62228639870114]
  We propose treating image completion as a directionless sequence-to-sequence prediction task. We employ a restrictive CNN with small and non-overlapping RF for token representation. In a second phase, to improve appearance consistency between visible and generated regions, a novel attention-aware layer (AAL) is introduced.
  arXiv  Detail & Related papers  (2021-04-02T01:42:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.