Toward Reliable AR-Guided Surgical Navigation: Interactive Deformation Modeling with Data-Driven Biomechanics and Prompts

• URL: http://arxiv.org/abs/2506.08048v2
• Date: Wed, 11 Jun 2025 03:34:08 GMT
• Title: Toward Reliable AR-Guided Surgical Navigation: Interactive Deformation Modeling with Data-Driven Biomechanics and Prompts
• Authors: Zheng Han, Jun Zhou, Jialun Pei, Jing Qin, Yingfang Fan, Qi Dou,
• Abstract summary: We propose a data-driven algorithm that preserves FEM-level accuracy while improving computational efficiency.<n>We introduce a novel human-in-the-loop mechanism into the deformation modeling process.<n>Our algorithm achieves a mean target registration error of 3.42 mm, surpassing state-of-the-art methods in volumetric accuracy.
• Score: 21.952265898720825
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In augmented reality (AR)-guided surgical navigation, preoperative organ models are superimposed onto the patient's intraoperative anatomy to visualize critical structures such as vessels and tumors. Accurate deformation modeling is essential to maintain the reliability of AR overlays by ensuring alignment between preoperative models and the dynamically changing anatomy. Although the finite element method (FEM) offers physically plausible modeling, its high computational cost limits intraoperative applicability. Moreover, existing algorithms often fail to handle large anatomical changes, such as those induced by pneumoperitoneum or ligament dissection, leading to inaccurate anatomical correspondences and compromised AR guidance. To address these challenges, we propose a data-driven biomechanics algorithm that preserves FEM-level accuracy while improving computational efficiency. In addition, we introduce a novel human-in-the-loop mechanism into the deformation modeling process. This enables surgeons to interactively provide prompts to correct anatomical misalignments, thereby incorporating clinical expertise and allowing the model to adapt dynamically to complex surgical scenarios. Experiments on a publicly available dataset demonstrate that our algorithm achieves a mean target registration error of 3.42 mm. Incorporating surgeon prompts through the interactive framework further reduces the error to 2.78 mm, surpassing state-of-the-art methods in volumetric accuracy. These results highlight the ability of our framework to deliver efficient and accurate deformation modeling while enhancing surgeon-algorithm collaboration, paving the way for safer and more reliable computer-assisted surgeries.

Related papers

• Semantic Segmentation for Preoperative Planning in Transcatheter Aortic Valve Replacement [61.573750959726475]
  We consider medical guidelines for preoperative planning of the transcatheter aortic valve replacement (TAVR) and identify tasks that may be supported via semantic segmentation models.<n>We first derive fine-grained TAVR-relevant pseudo-labels from coarse-grained anatomical information, in order to train segmentation models and quantify how well they are able to find these structures in the scans.
  arXiv  Detail & Related papers  (2025-07-22T13:24:45Z)
• ClipGS: Clippable Gaussian Splatting for Interactive Cinematic Visualization of Volumetric Medical Data [51.095474325541794]
  We introduce ClipGS, an innovative Gaussian splatting framework with the clipping plane supported, for interactive cinematic visualization of medical data.<n>We validate our method on five volumetric medical data, and reach an average 36.635 PSNR rendering quality with 156 FPS and 16.1 MB model size.
  arXiv  Detail & Related papers  (2025-07-09T08:24:28Z)
• Surgical Foundation Model Leveraging Compression and Entropy Maximization for Image-Guided Surgical Assistance [50.486523249499115]
  Real-time video understanding is critical to guide procedures in minimally invasive surgery (MIS)<n>We propose Compress-to-Explore (C2E), a novel self-supervised framework to learn compact, informative representations from surgical videos.<n>C2E uses entropy-maximizing decoders to compress images while preserving clinically relevant details, improving encoder performance without labeled data.
  arXiv  Detail & Related papers  (2025-05-16T14:02:24Z)
• Landmark-Free Preoperative-to-Intraoperative Registration in Laparoscopic Liver Resection [50.388465935739376]
  Liver registration by overlaying preoperative 3D models onto intraoperative 2D frames can assist surgeons in perceiving the spatial anatomy of the liver clearly for a higher surgical success rate.<n>Existing registration methods rely heavily on anatomical landmark-based, which encounter two major limitations.<n>We propose a landmark-free preoperative-to-intraoperative registration framework utilizing effective self-supervised learning.
  arXiv  Detail & Related papers  (2025-04-21T14:55:57Z)
• AI-Powered Automated Model Construction for Patient-Specific CFD Simulations of Aortic Flows [8.062885940500259]
  This study introduces a deep-learning framework that automates the creation of simulation-ready vascular models from medical images.<n>The proposed approach demonstrates state-of-the-art performance in segmentation and mesh quality while significantly reducing manual effort and processing time.
  arXiv  Detail & Related papers  (2025-03-16T14:18:25Z)
• A Review on Organ Deformation Modeling Approaches for Reliable Surgical Navigation using Augmented Reality [13.77243506924244]
  Augmented Reality (AR) holds the potential to revolutionize surgical procedures by allowing surgeons to visualize critical structures within the patient's body. Challenges arise from dynamic deformations of organs during surgery, making preoperative models inadequate for faithfully representing intraoperative anatomy. This review seeks to enhance the understanding of organ deformation modeling in AR-guided surgery, and discuss the potential topics for future advancements.
  arXiv  Detail & Related papers  (2024-08-05T14:03:17Z)
• Enhanced Knee Kinematics: Leveraging Deep Learning and Morphing Algorithms for 3D Implant Modeling [2.752817022620644]
  This study proposes a novel approach using machine learning algorithms and morphing techniques for precise 3D reconstruction of implanted knee models. A convolutional neural network is trained to automatically segment the femur contour of the implanted components. A morphing algorithm generates a personalized 3D model of the implanted knee joint.
  arXiv  Detail & Related papers  (2024-08-02T20:11:04Z)
• An Endoscopic Chisel: Intraoperative Imaging Carves 3D Anatomical Models [8.516340459721484]
  We propose a first vision-based approach to update the preoperative 3D anatomical model. Results show a decrease in error during surgical progression as opposed to increasing when no update is employed.
  arXiv  Detail & Related papers  (2024-02-19T05:06:52Z)
• Domain adaptation strategies for 3D reconstruction of the lumbar spine using real fluoroscopy data [9.21828361691977]
  This study tackles key obstacles in adopting surgical navigation in orthopedic surgeries. It shows an approach for generating 3D anatomical models of the spine from only a few fluoroscopic images. It achieved an 84% F1 score, matching the accuracy of our previous synthetic data-based research.
  arXiv  Detail & Related papers  (2024-01-29T10:22:45Z)
• Neural LerPlane Representations for Fast 4D Reconstruction of Deformable Tissues [52.886545681833596]
  LerPlane is a novel method for fast and accurate reconstruction of surgical scenes under a single-viewpoint setting. LerPlane treats surgical procedures as 4D volumes and factorizes them into explicit 2D planes of static and dynamic fields. LerPlane shares static fields, significantly reducing the workload of dynamic tissue modeling.
  arXiv  Detail & Related papers  (2023-05-31T14:38:35Z)
• Safe Deep RL for Intraoperative Planning of Pedicle Screw Placement [61.28459114068828]
  We propose an intraoperative planning approach for robotic spine surgery that leverages real-time observation for drill path planning based on Safe Deep Reinforcement Learning (DRL) Our approach was capable of achieving 90% bone penetration with respect to the gold standard (GS) drill planning.
  arXiv  Detail & Related papers  (2023-05-09T11:42:53Z)
• Real-time landmark detection for precise endoscopic submucosal dissection via shape-aware relation network [51.44506007844284]
  We propose a shape-aware relation network for accurate and real-time landmark detection in endoscopic submucosal dissection surgery. We first devise an algorithm to automatically generate relation keypoint heatmaps, which intuitively represent the prior knowledge of spatial relations among landmarks. We then develop two complementary regularization schemes to progressively incorporate the prior knowledge into the training process.
  arXiv  Detail & Related papers  (2021-11-08T07:57:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.