Explainable AI for Collaborative Assessment of 2D/3D Registration Quality

• URL: http://arxiv.org/abs/2507.17597v1
• Date: Wed, 23 Jul 2025 15:28:57 GMT
• Title: Explainable AI for Collaborative Assessment of 2D/3D Registration Quality
• Authors: Sue Min Cho, Alexander Do, Russell H. Taylor, Mathias Unberath,
• Abstract summary: We propose the first artificial intelligence framework trained specifically for 2D/3D registration quality verification.<n>Our explainable AI (XAI) approach aims to enhance informed decision-making for human operators.
• Score: 50.65650507103078
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: As surgery embraces digital transformation--integrating sophisticated imaging, advanced algorithms, and robotics to support and automate complex sub-tasks--human judgment of system correctness remains a vital safeguard for patient safety. This shift introduces new "operator-type" roles tasked with verifying complex algorithmic outputs, particularly at critical junctures of the procedure, such as the intermediary check before drilling or implant placement. A prime example is 2D/3D registration, a key enabler of image-based surgical navigation that aligns intraoperative 2D images with preoperative 3D data. Although registration algorithms have advanced significantly, they occasionally yield inaccurate results. Because even small misalignments can lead to revision surgery or irreversible surgical errors, there is a critical need for robust quality assurance. Current visualization-based strategies alone have been found insufficient to enable humans to reliably detect 2D/3D registration misalignments. In response, we propose the first artificial intelligence (AI) framework trained specifically for 2D/3D registration quality verification, augmented by explainability features that clarify the model's decision-making. Our explainable AI (XAI) approach aims to enhance informed decision-making for human operators by providing a second opinion together with a rationale behind it. Through algorithm-centric and human-centered evaluations, we systematically compare four conditions: AI-only, human-only, human-AI, and human-XAI. Our findings reveal that while explainability features modestly improve user trust and willingness to override AI errors, they do not exceed the standalone AI in aggregate performance. Nevertheless, future work extending both the algorithmic design and the human-XAI collaboration elements holds promise for more robust quality assurance of 2D/3D registration.

Related papers

• Visual Analytics for Explainable and Trustworthy Artificial Intelligence [2.1212179660694104]
  A key obstacle to AI adoption lies in the lack of transparency.<n>Many automated systems function as "black boxes," providing predictions without revealing the underlying processes.<n>Visual analytics (VA) provides a compelling solution by combining AI models with interactive visualizations.
  arXiv  Detail & Related papers  (2025-07-14T13:03:17Z)
• Quality Assessment and Distortion-aware Saliency Prediction for AI-Generated Omnidirectional Images [70.49595920462579]
  This work studies the quality assessment and distortion-aware saliency prediction problems for AIGODIs.<n>We propose two models with shared encoders based on the BLIP-2 model to evaluate the human visual experience and predict distortion-aware saliency for AI-generated omnidirectional images.
  arXiv  Detail & Related papers  (2025-06-27T05:36:04Z)
• Rapid patient-specific neural networks for intraoperative X-ray to volume registration [4.347837200266261]
  We present xvr, a fully automated framework for training patient-specific neural networks for 2D/3D registration.<n>xvr uses physics-based simulation to generate abundant high-quality training data from a patient's own preoperative volumetric imaging.<n>We perform the largest evaluation of a 2D/3D registration algorithm on real X-ray data to date and find that xvr robustly generalizes across a diverse dataset.
  arXiv  Detail & Related papers  (2025-03-20T16:33:45Z)
• Brain3D: Generating 3D Objects from fMRI [76.41771117405973]
  We design a novel 3D object representation learning method, Brain3D, that takes as input the fMRI data of a subject.<n>We show that our model captures the distinct functionalities of each region of human vision system.<n>Preliminary evaluations indicate that Brain3D can successfully identify the disordered brain regions in simulated scenarios.
  arXiv  Detail & Related papers  (2024-05-24T06:06:11Z)
• AiOS: All-in-One-Stage Expressive Human Pose and Shape Estimation [55.179287851188036]
  We introduce a novel all-in-one-stage framework, AiOS, for expressive human pose and shape recovery without an additional human detection step. We first employ a human token to probe a human location in the image and encode global features for each instance. Then, we introduce a joint-related token to probe the human joint in the image and encoder a fine-grained local feature.
  arXiv  Detail & Related papers  (2024-03-26T17:59:23Z)
• BinaryHPE: 3D Human Pose and Shape Estimation via Binarization [99.83378699846767]
  3D human pose and shape estimation (HPE) aims to reconstruct the 3D human body, face, and hands from a single image.<n>We propose BinaryHPE, a novel binarization method designed to estimate the 3D human body, face, and hands parameters efficiently.
  arXiv  Detail & Related papers  (2023-11-24T07:51:50Z)
• IM-IAD: Industrial Image Anomaly Detection Benchmark in Manufacturing [88.35145788575348]
  Image anomaly detection (IAD) is an emerging and vital computer vision task in industrial manufacturing. The lack of a uniform IM benchmark is hindering the development and usage of IAD methods in real-world applications. We construct a comprehensive image anomaly detection benchmark (IM-IAD), which includes 19 algorithms on seven major datasets.
  arXiv  Detail & Related papers  (2023-01-31T01:24:45Z)
• The Impact of Machine Learning on 2D/3D Registration for Image-guided Interventions: A Systematic Review and Perspective [6.669432838047949]
  Image-based navigation is widely considered the next frontier of minimally invasive surgery. 2D/3D registration is a technique to estimate the spatial relationships between 3D structures and 2D images. Recent advent of machine learning-based approaches to imaging problems holds promise for solving some of the notorious challenges in 2D/3D registration.
  arXiv  Detail & Related papers  (2021-08-04T18:31:29Z)
• Attention-Guided Version of 2D UNet for Automatic Brain Tumor Segmentation [2.371982686172067]
  Gliomas are the most common and aggressive among brain tumors, which cause a short life expectancy in their highest grade. Deep convolutional neural networks (DCNNs) have achieved a remarkable performance in brain tumor segmentation. However, this task is still difficult owing to high varying intensity and appearance of gliomas.
  arXiv  Detail & Related papers  (2020-04-04T20:09:06Z)
• Automatic Gesture Recognition in Robot-assisted Surgery with Reinforcement Learning and Tree Search [63.07088785532908]
  We propose a framework based on reinforcement learning and tree search for joint surgical gesture segmentation and classification. Our framework consistently outperforms the existing methods on the suturing task of JIGSAWS dataset in terms of accuracy, edit score and F1 score.
  arXiv  Detail & Related papers  (2020-02-20T13:12:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.