Multi-view X-ray Image Synthesis with Multiple Domain Disentanglement from CT Scans

• URL: http://arxiv.org/abs/2404.11889v2
• Date: Tue, 30 Jul 2024 23:50:55 GMT
• Title: Multi-view X-ray Image Synthesis with Multiple Domain Disentanglement from CT Scans
• Authors: Lixing Tan, Shuang Song, Kangneng Zhou, Chengbo Duan, Lanying Wang, Huayang Ren, Linlin Liu, Wei Zhang, Ruoxiu Xiao,
• Abstract summary: Over-dosed X-rays superimpose potential risks to human health to some extent. Data-driven algorithms from volume scans to X-ray images are restricted by the scarcity of paired X-ray and volume data. We propose CT2X-GAN to synthesize the X-ray images in an end-to-end manner using the content and style disentanglement from three different image domains.
• Score: 10.72672892416061
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: X-ray images play a vital role in the intraoperative processes due to their high resolution and fast imaging speed and greatly promote the subsequent segmentation, registration and reconstruction. However, over-dosed X-rays superimpose potential risks to human health to some extent. Data-driven algorithms from volume scans to X-ray images are restricted by the scarcity of paired X-ray and volume data. Existing methods are mainly realized by modelling the whole X-ray imaging procedure. In this study, we propose a learning-based approach termed CT2X-GAN to synthesize the X-ray images in an end-to-end manner using the content and style disentanglement from three different image domains. Our method decouples the anatomical structure information from CT scans and style information from unpaired real X-ray images/ digital reconstructed radiography (DRR) images via a series of decoupling encoders. Additionally, we introduce a novel consistency regularization term to improve the stylistic resemblance between synthesized X-ray images and real X-ray images. Meanwhile, we also impose a supervised process by computing the similarity of computed real DRR and synthesized DRR images. We further develop a pose attention module to fully strengthen the comprehensive information in the decoupled content code from CT scans, facilitating high-quality multi-view image synthesis in the lower 2D space. Extensive experiments were conducted on the publicly available CTSpine1K dataset and achieved 97.8350, 0.0842 and 3.0938 in terms of FID, KID and defined user-scored X-ray similarity, respectively. In comparison with 3D-aware methods ($\pi$-GAN, EG3D), CT2X-GAN is superior in improving the synthesis quality and realistic to the real X-ray images.

Related papers

• DiffuX2CT: Diffusion Learning to Reconstruct CT Images from Biplanar X-Rays [41.393567374399524]
  We propose DiffuX2CT, which models CT reconstruction from ultra-sparse X-rays as a conditional diffusion process. By doing so, DiffuX2CT achieves structure-controllable reconstruction, which enables 3D structural information to be recovered from 2D X-rays. As an extra contribution, we collect a real-world lumbar CT dataset, called LumbarV, as a new benchmark to verify the clinical significance and performance of CT reconstruction from X-rays.
  arXiv  Detail & Related papers  (2024-07-18T14:20:04Z)
• X-ray2CTPA: Generating 3D CTPA scans from 2D X-ray conditioning [24.233484690096898]
  Chest X-rays or chest radiography (CXR) enables limited imaging compared to computed tomography (CT) scans. CT scans entail higher costs, greater radiation exposure, and are less accessible than CXRs. In this work we explore cross-modal translation from a 2D low contrast-resolution X-ray input to a 3D high contrast and spatial-resolutionA scan.
  arXiv  Detail & Related papers  (2024-06-23T13:53:35Z)
• X-Ray: A Sequential 3D Representation For Generation [54.160173837582796]
  We introduce X-Ray, a novel 3D sequential representation inspired by x-ray scans. X-Ray transforms a 3D object into a series of surface frames at different layers, making it suitable for generating 3D models from images.
  arXiv  Detail & Related papers  (2024-04-22T16:40:11Z)
• Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis [88.86777314004044]
  We propose a 3D Gaussian splatting-based framework, namely X-Gaussian, for X-ray novel view visualization. Experiments show that our X-Gaussian outperforms state-of-the-art methods by 6.5 dB while enjoying less than 15% training time and over 73x inference speed.
  arXiv  Detail & Related papers  (2024-03-07T00:12:08Z)
• Multi-View Vertebra Localization and Identification from CT Images [57.56509107412658]
  We propose a multi-view vertebra localization and identification from CT images. We convert the 3D problem into a 2D localization and identification task on different views. Our method can learn the multi-view global information naturally.
  arXiv  Detail & Related papers  (2023-07-24T14:43:07Z)
• Perspective Projection-Based 3D CT Reconstruction from Biplanar X-rays [32.98966469644061]
  We propose PerX2CT, a novel CT reconstruction framework from X-ray. Our proposed method provides a different combination of features for each coordinate which implicitly allows the model to obtain information about the 3D location.
  arXiv  Detail & Related papers  (2023-03-09T14:45:25Z)
• X-Ray2EM: Uncertainty-Aware Cross-Modality Image Reconstruction from X-Ray to Electron Microscopy in Connectomics [55.6985304397137]
  We propose an uncertainty-aware 3D reconstruction model that translates X-ray images to EM-like images with enhanced membrane segmentation quality. This shows its potential for developing simpler, faster, and more accurate X-ray based connectomics pipelines.
  arXiv  Detail & Related papers  (2023-03-02T00:52:41Z)
• Cross-Modal Contrastive Learning for Abnormality Classification and Localization in Chest X-rays with Radiomics using a Feedback Loop [63.81818077092879]
  We propose an end-to-end semi-supervised cross-modal contrastive learning framework for medical images. We first apply an image encoder to classify the chest X-rays and to generate the image features. The radiomic features are then passed through another dedicated encoder to act as the positive sample for the image features generated from the same chest X-ray.
  arXiv  Detail & Related papers  (2021-04-11T09:16:29Z)
• XraySyn: Realistic View Synthesis From a Single Radiograph Through CT Priors [118.27130593216096]
  A radiograph visualizes the internal anatomy of a patient through the use of X-ray, which projects 3D information onto a 2D plane. To the best of our knowledge, this is the first work on radiograph view synthesis. We show that by gaining an understanding of radiography in 3D space, our method can be applied to radiograph bone extraction and suppression without groundtruth bone labels.
  arXiv  Detail & Related papers  (2020-12-04T05:08:53Z)
• End-To-End Convolutional Neural Network for 3D Reconstruction of Knee Bones From Bi-Planar X-Ray Images [6.645111950779666]
  We present an end-to-end Convolutional Neural Network (CNN) approach for 3D reconstruction of knee bones directly from two bi-planar X-ray images.
  arXiv  Detail & Related papers  (2020-04-02T08:37:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.