Benchmarking Encoder-Decoder Architectures for Biplanar X-ray to 3D Shape Reconstruction

• URL: http://arxiv.org/abs/2309.13587v2
• Date: Tue, 26 Sep 2023 04:56:07 GMT
• Title: Benchmarking Encoder-Decoder Architectures for Biplanar X-ray to 3D Shape Reconstruction
• Authors: Mahesh Shakya, Bishesh Khanal
• Abstract summary: We present an extensive evaluation of 8 2D-3D models on equal footing using 6 public datasets. Our results show that attention-based methods that capture global spatial relationships tend to perform better across all anatomies.
• Score: 1.1748284119769041
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Various deep learning models have been proposed for 3D bone shape reconstruction from two orthogonal (biplanar) X-ray images. However, it is unclear how these models compare against each other since they are evaluated on different anatomy, cohort and (often privately held) datasets. Moreover, the impact of the commonly optimized image-based segmentation metrics such as dice score on the estimation of clinical parameters relevant in 2D-3D bone shape reconstruction is not well known. To move closer toward clinical translation, we propose a benchmarking framework that evaluates tasks relevant to real-world clinical scenarios, including reconstruction of fractured bones, bones with implants, robustness to population shift, and error in estimating clinical parameters. Our open-source platform provides reference implementations of 8 models (many of whose implementations were not publicly available), APIs to easily collect and preprocess 6 public datasets, and the implementation of automatic clinical parameter and landmark extraction methods. We present an extensive evaluation of 8 2D-3D models on equal footing using 6 public datasets comprising images for four different anatomies. Our results show that attention-based methods that capture global spatial relationships tend to perform better across all anatomies and datasets; performance on clinically relevant subgroups may be overestimated without disaggregated reporting; ribs are substantially more difficult to reconstruct compared to femur, hip and spine; and the dice score improvement does not always bring a corresponding improvement in the automatic estimation of clinically relevant parameters.

Related papers

• Enhanced Knee Kinematics: Leveraging Deep Learning and Morphing Algorithms for 3D Implant Modeling [2.752817022620644]
  This study proposes a novel approach using machine learning algorithms and morphing techniques for precise 3D reconstruction of implanted knee models. A convolutional neural network is trained to automatically segment the femur contour of the implanted components. A morphing algorithm generates a personalized 3D model of the implanted knee joint.
  arXiv  Detail & Related papers  (2024-08-02T20:11:04Z)
• Anatomy-guided Pathology Segmentation [56.883822515800205]
  We develop a generalist segmentation model that combines anatomical and pathological information, aiming to enhance the segmentation accuracy of pathological features. Our Anatomy-Pathology Exchange (APEx) training utilizes a query-based segmentation transformer which decodes a joint feature space into query-representations for human anatomy. In doing so, we are able to report the best results across the board on FDG-PET-CT and Chest X-Ray pathology segmentation tasks with a margin of up to 3.3% as compared to strong baseline methods.
  arXiv  Detail & Related papers  (2024-07-08T11:44:15Z)
• Frenet-Serret Frame-based Decomposition for Part Segmentation of 3D Curvilinear Structures [26.53820792357073]
  Accurately segmenting 3D curvilinear structures in medical imaging remains challenging due to their complex geometry. We introduce a novel Frenet--Serret Frame-based Decomposition, which decomposes 3D curvilinear structures into a globally ( C2 ) continuous curve. Our experiments on DenSpineEM demonstrate exceptional cross-region and cross-species generalization.
  arXiv  Detail & Related papers  (2024-04-19T16:40:24Z)
• Domain adaptation strategies for 3D reconstruction of the lumbar spine using real fluoroscopy data [9.21828361691977]
  This study tackles key obstacles in adopting surgical navigation in orthopedic surgeries. It shows an approach for generating 3D anatomical models of the spine from only a few fluoroscopic images. It achieved an 84% F1 score, matching the accuracy of our previous synthetic data-based research.
  arXiv  Detail & Related papers  (2024-01-29T10:22:45Z)
• Unsupervised correspondence with combined geometric learning and imaging for radiotherapy applications [0.0]
  The aim of this study was to develop a model to accurately identify corresponding points between organ segmentations of different patients for radiotherapy applications. A model for simultaneous correspondence and estimation in 3D shapes was trained with head and neck organ segmentations from planning CT scans. We then extended the original model to incorporate imaging information using two approaches.
  arXiv  Detail & Related papers  (2023-09-25T16:29:18Z)
• Towards Unifying Anatomy Segmentation: Automated Generation of a Full-body CT Dataset via Knowledge Aggregation and Anatomical Guidelines [113.08940153125616]
  We generate a dataset of whole-body CT scans with $142$ voxel-level labels for 533 volumes providing comprehensive anatomical coverage. Our proposed procedure does not rely on manual annotation during the label aggregation stage. We release our trained unified anatomical segmentation model capable of predicting $142$ anatomical structures on CT data.
  arXiv  Detail & Related papers  (2023-07-25T09:48:13Z)
• 3DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentation [52.699139151447945]
  We propose a novel adaptation method for transferring the segment anything model (SAM) from 2D to 3D for promptable medical image segmentation. Our model can outperform domain state-of-the-art medical image segmentation models on 3 out of 4 tasks, specifically by 8.25%, 29.87%, and 10.11% for kidney tumor, pancreas tumor, colon cancer segmentation, and achieve similar performance for liver tumor segmentation.
  arXiv  Detail & Related papers  (2023-06-23T12:09:52Z)
• Medical Image Captioning via Generative Pretrained Transformers [57.308920993032274]
  We combine two language models, the Show-Attend-Tell and the GPT-3, to generate comprehensive and descriptive radiology records. The proposed model is tested on two medical datasets, the Open-I, MIMIC-CXR, and the general-purpose MS-COCO.
  arXiv  Detail & Related papers  (2022-09-28T10:27:10Z)
• A unified 3D framework for Organs at Risk Localization and Segmentation for Radiation Therapy Planning [56.52933974838905]
  Current medical workflow requires manual delineation of organs-at-risk (OAR) In this work, we aim to introduce a unified 3D pipeline for OAR localization-segmentation. Our proposed framework fully enables the exploitation of 3D context information inherent in medical imaging.
  arXiv  Detail & Related papers  (2022-03-01T17:08:41Z)
• Few-shot image segmentation for cross-institution male pelvic organs using registration-assisted prototypical learning [13.567073992605797]
  This work presents the first 3D few-shot interclass segmentation network for medical images. It uses a labelled multi-institution dataset from prostate cancer patients with eight regions of interest. A built-in registration mechanism can effectively utilise the prior knowledge of consistent anatomy between subjects.
  arXiv  Detail & Related papers  (2022-01-17T11:44:10Z)
• A Self-Supervised Deep Framework for Reference Bony Shape Estimation in Orthognathic Surgical Planning [55.30223654196882]
  A virtual orthognathic surgical planning involves simulating surgical corrections of jaw deformities on 3D facial bony shape models. A reference facial bony shape model representing normal anatomies can provide an objective guidance to improve planning accuracy. We propose a self-supervised deep framework to automatically estimate reference facial bony shape models.
  arXiv  Detail & Related papers  (2021-09-11T05:24:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.