Multimodal Learning With Intraoperative CBCT & Variably Aligned Preoperative CT Data To Improve Segmentation

• URL: http://arxiv.org/abs/2406.11650v2
• Date: Mon, 1 Jul 2024 09:57:32 GMT
• Title: Multimodal Learning With Intraoperative CBCT & Variably Aligned Preoperative CT Data To Improve Segmentation
• Authors: Maximilian E. Tschuchnig, Philipp Steininger, Michael Gadermayr,
• Abstract summary: Cone-beam computed tomography (CBCT) is an important tool facilitating computer aided interventions. While the degraded image quality can affect downstream segmentation, the availability of high quality, preoperative scans represents potential for improvements. We propose a multimodal learning method that fuses roughly aligned CBCT and CT scans and investigate the effect of CBCT quality and misalignment on the final segmentation performance.
• Score: 0.21847754147782888
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: Cone-beam computed tomography (CBCT) is an important tool facilitating computer aided interventions, despite often suffering from artifacts that pose challenges for accurate interpretation. While the degraded image quality can affect downstream segmentation, the availability of high quality, preoperative scans represents potential for improvements. Here we consider a setting where preoperative CT and intraoperative CBCT scans are available, however, the alignment (registration) between the scans is imperfect. We propose a multimodal learning method that fuses roughly aligned CBCT and CT scans and investigate the effect of CBCT quality and misalignment on the final segmentation performance. For that purpose, we make use of a synthetically generated data set containing real CT and synthetic CBCT volumes. As an application scenario, we focus on liver and liver tumor segmentation. We show that the fusion of preoperative CT and simulated, intraoperative CBCT mostly improves segmentation performance (compared to using intraoperative CBCT only) and that even clearly misaligned preoperative data has the potential to improve segmentation performance.

Related papers

• CBCTLiTS: A Synthetic, Paired CBCT/CT Dataset For Segmentation And Style Transfer [0.21847754147782888]
  We present CBCTLiTS, a synthetically generated, labelled CBCT dataset for segmentation with paired and aligned, high quality computed tomography data. The CBCT data is provided in 5 different levels of quality, reaching from a large number of projections with high visual quality to a small number of projections with severe artifacts.
  arXiv  Detail & Related papers  (2024-07-20T11:47:20Z)
• CT-based brain ventricle segmentation via diffusion Schrödinger Bridge without target domain ground truths [0.9720086191214947]
  Efficient and accurate brain ventricle segmentation from clinical CT scans is critical for emergency surgeries like ventriculostomy. We introduce a novel uncertainty-aware ventricle segmentation technique without the need of CT segmentation ground truths. Our method employs the diffusion Schr"odinger Bridge and an attention recurrent residual U-Net to capitalize on unpaired CT and MRI scans.
  arXiv  Detail & Related papers  (2024-05-28T15:17:58Z)
• CathFlow: Self-Supervised Segmentation of Catheters in Interventional Ultrasound Using Optical Flow and Transformers [66.15847237150909]
  We introduce a self-supervised deep learning architecture to segment catheters in longitudinal ultrasound images. The network architecture builds upon AiAReSeg, a segmentation transformer built with the Attention in Attention mechanism. We validated our model on a test dataset, consisting of unseen synthetic data and images collected from silicon aorta phantoms.
  arXiv  Detail & Related papers  (2024-03-21T15:13:36Z)
• A multi-channel cycleGAN for CBCT to CT synthesis [0.0]
  Image synthesis is used to generate synthetic CTs from on-treatment cone-beam CTs (CBCTs) Our contribution focuses on the second task, CBCT-to-sCT synthesis. By leveraging a multi-channel input to emphasize specific image features, our approach effectively addresses some of the challenges inherent in CBCT imaging.
  arXiv  Detail & Related papers  (2023-12-04T16:40:53Z)
• Reliable Joint Segmentation of Retinal Edema Lesions in OCT Images [55.83984261827332]
  In this paper, we propose a novel reliable multi-scale wavelet-enhanced transformer network. We develop a novel segmentation backbone that integrates a wavelet-enhanced feature extractor network and a multi-scale transformer module. Our proposed method achieves better segmentation accuracy with a high degree of reliability as compared to other state-of-the-art segmentation approaches.
  arXiv  Detail & Related papers  (2022-12-01T07:32:56Z)
• Data-Efficient Vision Transformers for Multi-Label Disease Classification on Chest Radiographs [55.78588835407174]
  Vision Transformers (ViTs) have not been applied to this task despite their high classification performance on generic images. ViTs do not rely on convolutions but on patch-based self-attention and in contrast to CNNs, no prior knowledge of local connectivity is present. Our results show that while the performance between ViTs and CNNs is on par with a small benefit for ViTs, DeiTs outperform the former if a reasonably large data set is available for training.
  arXiv  Detail & Related papers  (2022-08-17T09:07:45Z)
• Liver Segmentation using Turbolift Learning for CT and Cone-beam C-arm Perfusion Imaging [0.4073222202612759]
  Time separation technique (TST) was found to improve dynamic perfusion imaging of the liver using C-arm cone-beam computed tomography (CBCT) To apply TST using prior knowledge extracted from CT perfusion data, the liver should be accurately segmented from the CT scans. This research proposes Turbolift learning, which trains a modified version of the multi-scale Attention UNet on different liver segmentation tasks.
  arXiv  Detail & Related papers  (2022-07-20T19:38:50Z)
• CyTran: A Cycle-Consistent Transformer with Multi-Level Consistency for Non-Contrast to Contrast CT Translation [56.622832383316215]
  We propose a novel approach to translate unpaired contrast computed tomography (CT) scans to non-contrast CT scans. Our approach is based on cycle-consistent generative adversarial convolutional transformers, for short, CyTran. Our empirical results show that CyTran outperforms all competing methods.
  arXiv  Detail & Related papers  (2021-10-12T23:25:03Z)
• Multitask 3D CBCT-to-CT Translation and Organs-at-Risk Segmentation Using Physics-Based Data Augmentation [4.3971310109651665]
  In current clinical practice, noisy and artifact-ridden weekly cone-beam computed tomography (CBCT) images are only used for patient setup during radiotherapy. Treatment planning is done once at the beginning of the treatment using high-quality planning CT (pCT) images and manual contours for organs-at-risk (OARs) structures. If the quality of the weekly CBCT images can be improved while simultaneously segmenting OAR structures, this can provide critical information for adapting radiotherapy mid-treatment and for deriving biomarkers for treatment response.
  arXiv  Detail & Related papers  (2021-03-09T19:51:44Z)
• Synergistic Learning of Lung Lobe Segmentation and Hierarchical Multi-Instance Classification for Automated Severity Assessment of COVID-19 in CT Images [61.862364277007934]
  We propose a synergistic learning framework for automated severity assessment of COVID-19 in 3D CT images. A multi-task deep network (called M$2$UNet) is then developed to assess the severity of COVID-19 patients. Our M$2$UNet consists of a patch-level encoder, a segmentation sub-network for lung lobe segmentation, and a classification sub-network for severity assessment.
  arXiv  Detail & Related papers  (2020-05-08T03:16:15Z)
• Detecting Pancreatic Ductal Adenocarcinoma in Multi-phase CT Scans via Alignment Ensemble [77.5625174267105]
  Pancreatic ductal adenocarcinoma (PDAC) is one of the most lethal cancers among the population. Multiple phases provide more information than single phase, but they are unaligned and inhomogeneous in texture. We suggest an ensemble of all these alignments as a promising way to boost the performance of PDAC detection.
  arXiv  Detail & Related papers  (2020-03-18T19:06:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.