Augmented Reality-based Guidance with Deformable Registration in Head and Neck Tumor Resection

• URL: http://arxiv.org/abs/2503.08802v2
• Date: Fri, 31 Oct 2025 16:19:07 GMT
• Title: Augmented Reality-based Guidance with Deformable Registration in Head and Neck Tumor Resection
• Authors: Qingyun Yang, Fangjie Li, Jiayi Xu, Zixuan Liu, Sindhura Sridhar, Whitney Jin, Jennifer Du, Jon Heiselman, Michael Miga, Michael Topf, Jie Ying Wu,
• Abstract summary: We propose a novel deformable registration framework that incorporates thickness information into the registration process.<n>In tongue specimens, the proposed framework improved target registration error (TRE) by up to 33%.<n>We analyzed distinct deformation behaviors in different specimens, highlighting the need for tailored deformation strategies.
• Score: 3.0375728447182766
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Head and neck squamous cell carcinoma (HNSCC) has one of the highest rates of recurrence cases among solid malignancies. Recurrence rates can be reduced by improving positive margins localization. Frozen section analysis (FSA) of resected specimens is the gold standard for intraoperative margin assessment. However, because of the complex 3D anatomy and the significant shrinkage of resected specimens, accurate margin relocation from specimen back onto the resection site based on FSA results remains challenging. We propose a novel deformable registration framework that uses both the pre-resection upper surface and the post-resection site of the specimen to incorporate thickness information into the registration process. The proposed method significantly improves target registration error (TRE), demonstrating enhanced adaptability to thicker specimens. In tongue specimens, the proposed framework improved TRE by up to 33% as compared to prior deformable registration. Notably, tongue specimens exhibit complex 3D anatomies and hold the highest clinical significance compared to other head and neck specimens from the buccal and skin. We analyzed distinct deformation behaviors in different specimens, highlighting the need for tailored deformation strategies. To further aid intraoperative visualization, we also integrated this framework with an augmented reality-based auto-alignment system. The combined system can accurately and automatically overlay the deformed 3D specimen mesh with positive margin annotation onto the resection site. With a pilot study of the AR guided framework involving two surgeons, the integrated system improved the surgeons' average target relocation error from 9.8 cm to 4.8 cm.

Related papers

• Preoperative-to-intraoperative Liver Registration for Laparoscopic Surgery via Latent-Grounded Correspondence Constraints [51.7011449975586]
  Land-Reg is a deformable registration framework that learns latent-grounded 2D-3D landmark correspondences.<n>For rigid registration, Land-Reg embraces a Cross-modal Latent Alignment module.<n>An Uncertainty-enhanced Overlap Landmark Detector with similarity matching is proposed to robustly estimate explicit 2D-3D landmark correspondences.
  arXiv  Detail & Related papers  (2026-03-02T10:44:03Z)
• FoundationPose-Initialized 3D-2D Liver Registration for Surgical Augmented Reality [14.330143515294928]
  We integrate laparoscopic depth maps with a foundation pose estimator for camera-liver pose estimation.<n>On real patient data, the depth-augmented foundation pose approach achieved 9.91 mm mean registration error in 3 cases.<n>This pipeline achieves clinically relevant accuracy while offering a lightweight, engineering-friendly alternative to FE-based deformation.
  arXiv  Detail & Related papers  (2026-02-19T16:31:14Z)
• TempDiffReg: Temporal Diffusion Model for Non-Rigid 2D-3D Vascular Registration [14.97788368971466]
  The proposed method consistently outperforms state-of-the-art (SOTA) methods in both accuracy and anatomical plausibility.<n>It has the potential to assist less-experienced clinicians in safely performing complex TACE procedures.
  arXiv  Detail & Related papers  (2026-01-26T05:40:45Z)
• Landmark-Free Preoperative-to-Intraoperative Registration in Laparoscopic Liver Resection [50.388465935739376]
  Liver registration by overlaying preoperative 3D models onto intraoperative 2D frames can assist surgeons in perceiving the spatial anatomy of the liver clearly for a higher surgical success rate. Existing registration methods rely heavily on anatomical landmark-based, which encounter two major limitations. We propose a landmark-free preoperative-to-intraoperative registration framework utilizing effective self-supervised learning.
  arXiv  Detail & Related papers  (2025-04-21T14:55:57Z)
• 3D Acetabular Surface Reconstruction from 2D Pre-operative X-ray Images using SRVF Elastic Registration and Deformation Graph [15.26681988459618]
  This paper proposes a novel framework that integrates square-root velocity function (SRVF)-based elastic shape registration technique. It reconstructs the 3D articular surface of the acetabulum by fusing multiple views of 2D pre-operative pelvic X-ray images and a hemispherical surface model.
  arXiv  Detail & Related papers  (2025-03-28T06:47:32Z)
• Submillimeter-Accurate 3D Lumbar Spine Reconstruction from Biplanar X-Ray Images: Incorporating a Multi-Task Network and Landmark-Weighted Loss [5.514961917579977]
  This study presents a novel, fully automatic framework for high-precision 3D reconstruction from biplanar X-ray images.<n>The core of this method involves a novel multi-task deep learning network that simultaneously performs lumbar decomposition and landmark detection.<n>It sets a new benchmark by achieving sub-millimeter accuracy and completes the full reconstruction and measurement workflow in under 20 seconds.
  arXiv  Detail & Related papers  (2025-03-18T15:00:39Z)
• Fine-Tuning TransMorph with Gradient Correlation for Anatomical Alignment [0.6359529834975265]
  Unsupervised deep learning is a promising method in brain MRI registration to reduce the reliance on anatomical labels.<n>For the Learn2Reg2024 LUMIR challenge, we propose fine-tuning of the pre-trained TransMorph model to improve convergence stability as well as the deformation.
  arXiv  Detail & Related papers  (2024-12-30T09:32:04Z)
• Epicardium Prompt-guided Real-time Cardiac Ultrasound Frame-to-volume Registration [50.602074919305636]
  This paper introduces a lightweight end-to-end Cardiac Ultrasound frame-to-volume Registration network, termed CU-Reg.<n>We use epicardium prompt-guided anatomical clues to reinforce the interaction of 2D sparse and 3D dense features, followed by a voxel-wise local-global aggregation of enhanced features.
  arXiv  Detail & Related papers  (2024-06-20T17:47:30Z)
• On the Localization of Ultrasound Image Slices within Point Distribution Models [84.27083443424408]
  Thyroid disorders are most commonly diagnosed using high-resolution Ultrasound (US) Longitudinal tracking is a pivotal diagnostic protocol for monitoring changes in pathological thyroid morphology. We present a framework for automated US image slice localization within a 3D shape representation.
  arXiv  Detail & Related papers  (2023-09-01T10:10:46Z)
• Abdominal organ segmentation via deep diffeomorphic mesh deformations [5.4173776411667935]
  Abdominal organ segmentation from CT and MRI is an essential prerequisite for surgical planning and computer-aided navigation systems. We employ template-based mesh reconstruction methods for joint liver, kidney, pancreas, and spleen segmentation. The resulting method, UNetFlow, generalizes well to all four organs and can be easily fine-tuned on new data.
  arXiv  Detail & Related papers  (2023-06-27T14:41:18Z)
• 3DSAM-adapter: Holistic adaptation of SAM from 2D to 3D for promptable tumor segmentation [52.699139151447945]
  We propose a novel adaptation method for transferring the segment anything model (SAM) from 2D to 3D for promptable medical image segmentation. Our model can outperform domain state-of-the-art medical image segmentation models on 3 out of 4 tasks, specifically by 8.25%, 29.87%, and 10.11% for kidney tumor, pancreas tumor, colon cancer segmentation, and achieve similar performance for liver tumor segmentation.
  arXiv  Detail & Related papers  (2023-06-23T12:09:52Z)
• Adversarial Transformer for Repairing Human Airway Segmentation [7.176060570019899]
  This paper presents a patch-scale adversarial-based refinement network that takes in preliminary segmentation along with original CT images and outputs a refined mask of the airway structure. The results are quantitatively evaluated by seven metrics and achieved more than a 15% rise in detected length ratio and detected branch ratio.
  arXiv  Detail & Related papers  (2022-10-21T15:20:08Z)
• Explainable multiple abnormality classification of chest CT volumes with AxialNet and HiResCAM [89.2175350956813]
  We introduce the challenging new task of explainable multiple abnormality classification in volumetric medical images. We propose a multiple instance learning convolutional neural network, AxialNet, that allows identification of top slices for each abnormality. We then aim to improve the model's learning through a novel mask loss that leverages HiResCAM and 3D allowed regions.
  arXiv  Detail & Related papers  (2021-11-24T01:14:33Z)
• A Self-Supervised Deep Framework for Reference Bony Shape Estimation in Orthognathic Surgical Planning [55.30223654196882]
  A virtual orthognathic surgical planning involves simulating surgical corrections of jaw deformities on 3D facial bony shape models. A reference facial bony shape model representing normal anatomies can provide an objective guidance to improve planning accuracy. We propose a self-supervised deep framework to automatically estimate reference facial bony shape models.
  arXiv  Detail & Related papers  (2021-09-11T05:24:40Z)
• Enhanced 3D Myocardial Strain Estimation from Multi-View 2D CMR Imaging [0.0]
  We propose an enhanced 3D myocardial strain estimation procedure, which combines complementary displacement information from multiple orientations of a single imaging modality (untagged CMR SSFP images) We register the sets of short-axis, four-chamber and two-chamber views via a 2D non-rigid registration algorithm implemented in a commercial software (Segment, Medviso) We then create a series of interpolating functions for the three directions of motion and use them to deform a tetrahedral mesh representation of a patient-specific left ventricle.
  arXiv  Detail & Related papers  (2020-09-25T22:47:50Z)
• Deep Volumetric Universal Lesion Detection using Light-Weight Pseudo 3D Convolution and Surface Point Regression [23.916776570010285]
  Computer-aided lesion/significant-findings detection techniques are at the core of medical imaging. We propose a novel deep anchor-free one-stage VULD framework that incorporates (1) P3DC operators to recycle the architectural configurations and pre-trained weights from the off-the-shelf 2D networks. New SPR method to effectively regress the 3D lesion spatial extents by pinpointing their representative key points on lesion surfaces.
  arXiv  Detail & Related papers  (2020-08-30T19:42:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.