Benchmarking Encoder-Decoder Architectures for Biplanar X-ray to 3D
Shape Reconstruction
- URL: http://arxiv.org/abs/2309.13587v2
- Date: Tue, 26 Sep 2023 04:56:07 GMT
- Title: Benchmarking Encoder-Decoder Architectures for Biplanar X-ray to 3D
Shape Reconstruction
- Authors: Mahesh Shakya, Bishesh Khanal
- Abstract summary: We present an extensive evaluation of 8 2D-3D models on equal footing using 6 public datasets.
Our results show that attention-based methods that capture global spatial relationships tend to perform better across all anatomies.
- Score: 1.1748284119769041
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Various deep learning models have been proposed for 3D bone shape
reconstruction from two orthogonal (biplanar) X-ray images. However, it is
unclear how these models compare against each other since they are evaluated on
different anatomy, cohort and (often privately held) datasets. Moreover, the
impact of the commonly optimized image-based segmentation metrics such as dice
score on the estimation of clinical parameters relevant in 2D-3D bone shape
reconstruction is not well known. To move closer toward clinical translation,
we propose a benchmarking framework that evaluates tasks relevant to real-world
clinical scenarios, including reconstruction of fractured bones, bones with
implants, robustness to population shift, and error in estimating clinical
parameters. Our open-source platform provides reference implementations of 8
models (many of whose implementations were not publicly available), APIs to
easily collect and preprocess 6 public datasets, and the implementation of
automatic clinical parameter and landmark extraction methods. We present an
extensive evaluation of 8 2D-3D models on equal footing using 6 public datasets
comprising images for four different anatomies. Our results show that
attention-based methods that capture global spatial relationships tend to
perform better across all anatomies and datasets; performance on clinically
relevant subgroups may be overestimated without disaggregated reporting; ribs
are substantially more difficult to reconstruct compared to femur, hip and
spine; and the dice score improvement does not always bring a corresponding
improvement in the automatic estimation of clinically relevant parameters.
Related papers
- Anatomy-guided Pathology Segmentation [56.883822515800205]
We develop a generalist segmentation model that combines anatomical and pathological information, aiming to enhance the segmentation accuracy of pathological features.
Our Anatomy-Pathology Exchange (APEx) training utilizes a query-based segmentation transformer which decodes a joint feature space into query-representations for human anatomy.
In doing so, we are able to report the best results across the board on FDG-PET-CT and Chest X-Ray pathology segmentation tasks with a margin of up to 3.3% as compared to strong baseline methods.
arXiv Detail & Related papers (2024-07-08T11:44:15Z) - FreSeg: Frenet-Frame-based Part Segmentation for 3D Curvilinear Structures [26.814165975617136]
Part segmentation is a crucial task for 3D curvilinear structures like neuron dendrites and blood vessels.
We propose FreSeg, a framework of part segmentation tasks for 3D curvilinear structures.
arXiv Detail & Related papers (2024-04-19T16:40:24Z) - Domain adaptation strategies for 3D reconstruction of the lumbar spine using real fluoroscopy data [9.21828361691977]
This study tackles key obstacles in adopting surgical navigation in orthopedic surgeries.
It shows an approach for generating 3D anatomical models of the spine from only a few fluoroscopic images.
It achieved an 84% F1 score, matching the accuracy of our previous synthetic data-based research.
arXiv Detail & Related papers (2024-01-29T10:22:45Z) - Unsupervised correspondence with combined geometric learning and imaging
for radiotherapy applications [0.0]
The aim of this study was to develop a model to accurately identify corresponding points between organ segmentations of different patients for radiotherapy applications.
A model for simultaneous correspondence and estimation in 3D shapes was trained with head and neck organ segmentations from planning CT scans.
We then extended the original model to incorporate imaging information using two approaches.
arXiv Detail & Related papers (2023-09-25T16:29:18Z) - Towards Unifying Anatomy Segmentation: Automated Generation of a
Full-body CT Dataset via Knowledge Aggregation and Anatomical Guidelines [113.08940153125616]
We generate a dataset of whole-body CT scans with $142$ voxel-level labels for 533 volumes providing comprehensive anatomical coverage.
Our proposed procedure does not rely on manual annotation during the label aggregation stage.
We release our trained unified anatomical segmentation model capable of predicting $142$ anatomical structures on CT data.
arXiv Detail & Related papers (2023-07-25T09:48:13Z) - Med-Query: Steerable Parsing of 9-DoF Medical Anatomies with Query
Embedding [15.98677736544302]
We propose a steerable, robust, and efficient computing framework for detection, identification, and segmentation of anatomies in 3D medical data.
Considering complicated shapes, sizes and orientations of anatomies, we present the nine degrees-of-freedom (9-DoF) pose estimation solution in full 3D space.
We have validated the proposed method on three medical imaging parsing tasks of ribs, spine, and abdominal organs.
arXiv Detail & Related papers (2022-12-05T04:04:21Z) - Medical Image Captioning via Generative Pretrained Transformers [57.308920993032274]
We combine two language models, the Show-Attend-Tell and the GPT-3, to generate comprehensive and descriptive radiology records.
The proposed model is tested on two medical datasets, the Open-I, MIMIC-CXR, and the general-purpose MS-COCO.
arXiv Detail & Related papers (2022-09-28T10:27:10Z) - A unified 3D framework for Organs at Risk Localization and Segmentation
for Radiation Therapy Planning [56.52933974838905]
Current medical workflow requires manual delineation of organs-at-risk (OAR)
In this work, we aim to introduce a unified 3D pipeline for OAR localization-segmentation.
Our proposed framework fully enables the exploitation of 3D context information inherent in medical imaging.
arXiv Detail & Related papers (2022-03-01T17:08:41Z) - Few-shot image segmentation for cross-institution male pelvic organs
using registration-assisted prototypical learning [13.567073992605797]
This work presents the first 3D few-shot interclass segmentation network for medical images.
It uses a labelled multi-institution dataset from prostate cancer patients with eight regions of interest.
A built-in registration mechanism can effectively utilise the prior knowledge of consistent anatomy between subjects.
arXiv Detail & Related papers (2022-01-17T11:44:10Z) - A Self-Supervised Deep Framework for Reference Bony Shape Estimation in
Orthognathic Surgical Planning [55.30223654196882]
A virtual orthognathic surgical planning involves simulating surgical corrections of jaw deformities on 3D facial bony shape models.
A reference facial bony shape model representing normal anatomies can provide an objective guidance to improve planning accuracy.
We propose a self-supervised deep framework to automatically estimate reference facial bony shape models.
arXiv Detail & Related papers (2021-09-11T05:24:40Z) - Monocular Human Pose and Shape Reconstruction using Part Differentiable
Rendering [53.16864661460889]
Recent works succeed in regression-based methods which estimate parametric models directly through a deep neural network supervised by 3D ground truth.
In this paper, we introduce body segmentation as critical supervision.
To improve the reconstruction with part segmentation, we propose a part-level differentiable part that enables part-based models to be supervised by part segmentation.
arXiv Detail & Related papers (2020-03-24T14:25:46Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.