RT-SRTS: Angle-Agnostic Real-Time Simultaneous 3D Reconstruction and Tumor Segmentation from Single X-Ray Projection

• URL: http://arxiv.org/abs/2310.08080v2
• Date: Thu, 28 Mar 2024 09:20:33 GMT
• Title: RT-SRTS: Angle-Agnostic Real-Time Simultaneous 3D Reconstruction and Tumor Segmentation from Single X-Ray Projection
• Abstract summary: RT-SRTS is proposed which integrates 3D imaging and tumor segmentation into one network based on multi-task learning (MTL) The proposed method was evaluated on fifteen patient cases and compared with three state-of-the-art methods. Simultaneous reconstruction and segmentation can be completed in approximately 70 ms, significantly faster than the required time threshold for real-time tumor tracking.
• Score: 4.992840879935547
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Radiotherapy is one of the primary treatment methods for tumors, but the organ movement caused by respiration limits its accuracy. Recently, 3D imaging from a single X-ray projection has received extensive attention as a promising approach to address this issue. However, current methods can only reconstruct 3D images without directly locating the tumor and are only validated for fixed-angle imaging, which fails to fully meet the requirements of motion control in radiotherapy. In this study, a novel imaging method RT-SRTS is proposed which integrates 3D imaging and tumor segmentation into one network based on multi-task learning (MTL) and achieves real-time simultaneous 3D reconstruction and tumor segmentation from a single X-ray projection at any angle. Furthermore, the attention enhanced calibrator (AEC) and uncertain-region elaboration (URE) modules have been proposed to aid feature extraction and improve segmentation accuracy. The proposed method was evaluated on fifteen patient cases and compared with three state-of-the-art methods. It not only delivers superior 3D reconstruction but also demonstrates commendable tumor segmentation results. Simultaneous reconstruction and segmentation can be completed in approximately 70 ms, significantly faster than the required time threshold for real-time tumor tracking. The efficacies of both AEC and URE have also been validated in ablation studies. The code of work is available at https://github.com/ZywooSimple/RT-SRTS.

Related papers

• AI Approach for MRI-only Full-Spine Vertebral Segmentation and 3D Reconstruction in Paediatric Scoliosis [0.0]
  MRI is preferred over CT in paediatric imaging because it avoids ionising radiation.<n>Its use in spine deformity assessment is largely limited by the lack of automated, high-resolution 3D bony reconstruction.<n>This study introduces an AI framework that enables fully automated thoracolumbar spine (T1-L5) segmentation and 3D reconstruction from MRI alone.
  arXiv  Detail & Related papers  (2026-04-20T05:57:06Z)
• DRBD-Mamba for Robust and Efficient Brain Tumor Segmentation with Analytical Insights [54.87947751720332]
  Accurate brain tumor segmentation is significant for clinical diagnosis and treatment.<n>Mamba-based State Space Models have demonstrated promising performance.<n>We propose a dual-resolution bi-directional Mamba that captures multi-scale long-range dependencies with minimal computational overhead.
  arXiv  Detail & Related papers  (2025-10-16T07:31:21Z)
• Semantic 3D Reconstructions with SLAM for Central Airway Obstruction [4.925599926643841]
  Central airway obstruction (CAO) is a life-threatening condition with increasing incidence.<n>Traditional treatment methods such as bronchoscopy and electrocautery can be used to remove the tumor completely.<n>Recent advances allow robotic interventions with lesser risk.<n>We present a novel pipeline that enables real-time, semantically informed 3D reconstructions of the central airway using monocular endoscopic video.
  arXiv  Detail & Related papers  (2025-09-16T21:14:16Z)
• Submillimeter-Accurate 3D Lumbar Spine Reconstruction from Biplanar X-Ray Images: Incorporating a Multi-Task Network and Landmark-Weighted Loss [5.514961917579977]
  This study presents a novel, fully automatic framework for high-precision 3D reconstruction from biplanar X-ray images.<n>The core of this method involves a novel multi-task deep learning network that simultaneously performs lumbar decomposition and landmark detection.<n>It sets a new benchmark by achieving sub-millimeter accuracy and completes the full reconstruction and measurement workflow in under 20 seconds.
  arXiv  Detail & Related papers  (2025-03-18T15:00:39Z)
• TomoGRAF: A Robust and Generalizable Reconstruction Network for Single-View Computed Tomography [3.1209855614927275]
  Traditional analytical/iterative CT reconstruction algorithms require hundreds of angular data samplings. We develop a novel TomoGRAF framework incorporating the unique X-ray transportation physics to reconstruct high-quality 3D volumes.
  arXiv  Detail & Related papers  (2024-11-12T20:07:59Z)
• C^2RV: Cross-Regional and Cross-View Learning for Sparse-View CBCT Reconstruction [17.54830070112685]
  Cone beam computed tomography (CBCT) is an important imaging technology widely used in medical scenarios. CBCT reconstruction is more challenging due to the increased dimensionality caused by the measurement process based on cone-shaped X-ray beams. We propose C2RV by leveraging explicit multi-scale volumetric representations to enable cross-regional learning in the 3D space.
  arXiv  Detail & Related papers  (2024-06-06T09:37:56Z)
• Exploration of Multi-Scale Image Fusion Systems in Intelligent Medical Image Analysis [3.881664394416534]
  It is necessary to perform automatic segmentation of brain tumors on MRI images. This project intends to build an MRI algorithm based on U-Net.
  arXiv  Detail & Related papers  (2024-05-23T04:33:12Z)
• 3DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentation [52.699139151447945]
  We propose a novel adaptation method for transferring the segment anything model (SAM) from 2D to 3D for promptable medical image segmentation. Our model can outperform domain state-of-the-art medical image segmentation models on 3 out of 4 tasks, specifically by 8.25%, 29.87%, and 10.11% for kidney tumor, pancreas tumor, colon cancer segmentation, and achieve similar performance for liver tumor segmentation.
  arXiv  Detail & Related papers  (2023-06-23T12:09:52Z)
• 3D Brainformer: 3D Fusion Transformer for Brain Tumor Segmentation [6.127298607534532]
  Deep learning has recently emerged to improve brain tumor segmentation. Transformers have been leveraged to address the limitations of convolutional networks. We propose a 3D Transformer-based segmentation approach.
  arXiv  Detail & Related papers  (2023-04-28T02:11:29Z)
• X-Ray2EM: Uncertainty-Aware Cross-Modality Image Reconstruction from X-Ray to Electron Microscopy in Connectomics [55.6985304397137]
  We propose an uncertainty-aware 3D reconstruction model that translates X-ray images to EM-like images with enhanced membrane segmentation quality. This shows its potential for developing simpler, faster, and more accurate X-ray based connectomics pipelines.
  arXiv  Detail & Related papers  (2023-03-02T00:52:41Z)
• Attentive Symmetric Autoencoder for Brain MRI Segmentation [56.02577247523737]
  We propose a novel Attentive Symmetric Auto-encoder based on Vision Transformer (ViT) for 3D brain MRI segmentation tasks. In the pre-training stage, the proposed auto-encoder pays more attention to reconstruct the informative patches according to the gradient metrics. Experimental results show that our proposed attentive symmetric auto-encoder outperforms the state-of-the-art self-supervised learning methods and medical image segmentation models.
  arXiv  Detail & Related papers  (2022-09-19T09:43:19Z)
• A Long Short-term Memory Based Recurrent Neural Network for Interventional MRI Reconstruction [50.1787181309337]
  We propose a convolutional long short-term memory (Conv-LSTM) based recurrent neural network (RNN), or ConvLR, to reconstruct interventional images with golden-angle radial sampling. The proposed algorithm has the potential to achieve real-time i-MRI for DBS and can be used for general purpose MR-guided intervention.
  arXiv  Detail & Related papers  (2022-03-28T14:03:45Z)
• A unified 3D framework for Organs at Risk Localization and Segmentation for Radiation Therapy Planning [56.52933974838905]
  Current medical workflow requires manual delineation of organs-at-risk (OAR) In this work, we aim to introduce a unified 3D pipeline for OAR localization-segmentation. Our proposed framework fully enables the exploitation of 3D context information inherent in medical imaging.
  arXiv  Detail & Related papers  (2022-03-01T17:08:41Z)
• EMT-NET: Efficient multitask network for computer-aided diagnosis of breast cancer [58.720142291102135]
  We propose an efficient and light-weighted learning architecture to classify and segment breast tumors simultaneously. We incorporate a segmentation task into a tumor classification network, which makes the backbone network learn representations focused on tumor regions. The accuracy, sensitivity, and specificity of tumor classification is 88.6%, 94.1%, and 85.3%, respectively.
  arXiv  Detail & Related papers  (2022-01-13T05:24:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.