Simulation-Driven Training of Vision Transformers Enabling Metal Segmentation in X-Ray Images

• URL: http://arxiv.org/abs/2203.09207v1
• Date: Thu, 17 Mar 2022 09:58:58 GMT
• Title: Simulation-Driven Training of Vision Transformers Enabling Metal Segmentation in X-Ray Images
• Authors: Fuxin Fan, Ludwig Ritschl, Marcel Beister, Ramyar Biniazan, Bj\"orn Kreher, Tristan M. Gottschalk, Steffen Kappler, Andreas Maier
• Abstract summary: This study proposes to generate simulated X-ray images based on CT data sets combined with computer aided design (CAD) implants. The metal segmentation in CBCT projections serves as a prerequisite for metal artifact avoidance and reduction algorithms. Our study indicates that the CAD model-based data generation has high flexibility and could be a way to overcome the problem of shortage in clinical data sampling and labelling.
• Score: 6.416928579907334
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In several image acquisition and processing steps of X-ray radiography, knowledge of the existence of metal implants and their exact position is highly beneficial (e.g. dose regulation, image contrast adjustment). Another application which would benefit from an accurate metal segmentation is cone beam computed tomography (CBCT) which is based on 2D X-ray projections. Due to the high attenuation of metals, severe artifacts occur in the 3D X-ray acquisitions. The metal segmentation in CBCT projections usually serves as a prerequisite for metal artifact avoidance and reduction algorithms. Since the generation of high quality clinical training is a constant challenge, this study proposes to generate simulated X-ray images based on CT data sets combined with self-designed computer aided design (CAD) implants and make use of convolutional neural network (CNN) and vision transformer (ViT) for metal segmentation. Model test is performed on accurately labeled X-ray test datasets obtained from specimen scans. The CNN encoder-based network like U-Net has limited performance on cadaver test data with an average dice score below 0.30, while the metal segmentation transformer with dual decoder (MST-DD) shows high robustness and generalization on the segmentation task, with an average dice score of 0.90. Our study indicates that the CAD model-based data generation has high flexibility and could be a way to overcome the problem of shortage in clinical data sampling and labelling. Furthermore, the MST-DD approach generates a more reliable neural network in case of training on simulated data.

Related papers

• X-Ray to CT Rigid Registration Using Scene Coordinate Regression [1.1687067206676627]
  This paper proposes a fully automatic registration method that is robust to extreme viewpoints. It is based on a fully convolutional neural network (CNN) that regresses the overlapping coordinates for a given X-ray image. The proposed method achieved an average mean target registration error (mTRE) of 3.79 mm in the 50th percentile of the simulated test dataset and projected mTRE of 9.65 mm in the 50th percentile of real fluoroscopic images for pelvis registration.
  arXiv  Detail & Related papers  (2023-11-25T17:48:46Z)
• 2DeteCT -- A large 2D expandable, trainable, experimental Computed Tomography dataset for machine learning [1.0266286487433585]
  We provide a versatile, open 2D fan-beam CT dataset suitable for developing machine learning techniques. A diverse mix of samples with high natural variability in shape and density was scanned slice-by-slice. We provide raw projection data, reference reconstructions and segmentations based on an open-source data processing pipeline.
  arXiv  Detail & Related papers  (2023-06-09T14:02:53Z)
• Geometry-Aware Attenuation Learning for Sparse-View CBCT Reconstruction [53.93674177236367]
  Cone Beam Computed Tomography (CBCT) plays a vital role in clinical imaging. Traditional methods typically require hundreds of 2D X-ray projections to reconstruct a high-quality 3D CBCT image. This has led to a growing interest in sparse-view CBCT reconstruction to reduce radiation doses. We introduce a novel geometry-aware encoder-decoder framework to solve this problem.
  arXiv  Detail & Related papers  (2023-03-26T14:38:42Z)
• Orientation-Shared Convolution Representation for CT Metal Artifact Learning [63.67718355820655]
  During X-ray computed tomography (CT) scanning, metallic implants carrying with patients often lead to adverse artifacts. Existing deep-learning-based methods have gained promising reconstruction performance. We propose an orientation-shared convolution representation strategy to adapt the physical prior structures of artifacts.
  arXiv  Detail & Related papers  (2022-12-26T13:56:12Z)
• Metal artifact correction in cone beam computed tomography using synthetic X-ray data [0.0]
  Metal implants inserted into the anatomy cause severe artifacts in reconstructed images. One approach is to use a deep learning method to segment metals in the projections. We show that simulations with relatively small number of photons are suitable for the metal segmentation task.
  arXiv  Detail & Related papers  (2022-08-17T13:31:38Z)
• Data-Efficient Vision Transformers for Multi-Label Disease Classification on Chest Radiographs [55.78588835407174]
  Vision Transformers (ViTs) have not been applied to this task despite their high classification performance on generic images. ViTs do not rely on convolutions but on patch-based self-attention and in contrast to CNNs, no prior knowledge of local connectivity is present. Our results show that while the performance between ViTs and CNNs is on par with a small benefit for ViTs, DeiTs outperform the former if a reasonably large data set is available for training.
  arXiv  Detail & Related papers  (2022-08-17T09:07:45Z)
• Two-Stream Graph Convolutional Network for Intra-oral Scanner Image Segmentation [133.02190910009384]
  We propose a two-stream graph convolutional network (i.e., TSGCN) to handle inter-view confusion between different raw attributes. Our TSGCN significantly outperforms state-of-the-art methods in 3D tooth (surface) segmentation.
  arXiv  Detail & Related papers  (2022-04-19T10:41:09Z)
• Metal Artifact Reduction with Intra-Oral Scan Data for 3D Low Dose Maxillofacial CBCT Modeling [0.7444835592104696]
  A two-stage metal artifact reduction method is proposed for accurate 3D low-dose maxillofacial CBCT modeling. In the first stage, an image-to-image deep learning network is employed to mitigate metal-related artifacts. In the second stage, a 3D maxillofacial model is constructed by segmenting the bones from the dental CBCT image corrected.
  arXiv  Detail & Related papers  (2022-02-08T00:24:41Z)
• Many-to-One Distribution Learning and K-Nearest Neighbor Smoothing for Thoracic Disease Identification [83.6017225363714]
  deep learning has become the most powerful computer-aided diagnosis technology for improving disease identification performance. For chest X-ray imaging, annotating large-scale data requires professional domain knowledge and is time-consuming. In this paper, we propose many-to-one distribution learning (MODL) and K-nearest neighbor smoothing (KNNS) methods to improve a single model's disease identification performance.
  arXiv  Detail & Related papers  (2021-02-26T02:29:30Z)
• A Learning-based Method for Online Adjustment of C-arm Cone-Beam CT Source Trajectories for Artifact Avoidance [47.345403652324514]
  The reconstruction quality attainable with commercial CBCT devices is insufficient due to metal artifacts in the presence of pedicle screws. We propose to adjust the C-arm CBCT source trajectory during the scan to optimize reconstruction quality with respect to a certain task. We demonstrate that convolutional neural networks trained on realistically simulated data are capable of predicting quality metrics that enable scene-specific adjustments of the CBCT source trajectory.
  arXiv  Detail & Related papers  (2020-08-14T09:23:50Z)
• End-To-End Convolutional Neural Network for 3D Reconstruction of Knee Bones From Bi-Planar X-Ray Images [6.645111950779666]
  We present an end-to-end Convolutional Neural Network (CNN) approach for 3D reconstruction of knee bones directly from two bi-planar X-ray images.
  arXiv  Detail & Related papers  (2020-04-02T08:37:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.