Metric Surface Reconstruction of Neurosurgical Scenes from Monocular Operating Microscope Images and Microscope Pose

• URL: http://arxiv.org/abs/2607.22773v1
• Date: Fri, 24 Jul 2026 06:57:39 GMT
• Title: Metric Surface Reconstruction of Neurosurgical Scenes from Monocular Operating Microscope Images and Microscope Pose
• Abstract summary: We evaluated whether metric 3D geometry of neurosurgical operative exposure can be recovered from microscope images combined with pose data.<n>In a phantom-based laboratory study, two aneurysm training phantoms were imaged with a ZEISS Pentero 800 microscope integrated with Brainlab Cranial Navigation.<n>For phantom A, representing a deeper surgical corridor, reconstruction accuracy ranged from 1.95 $pm$ 1.70 mm to 2.33 $pm$ 2.15 mm.<n>For phantom B, representing a directly exposed surface, accuracy ranged from 1.02 $pm$ 0.93 mm to 1.52 $pm$ 1.2
• Score: 0.0817636641432964
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Objective: We evaluated whether metric 3D geometry of neurosurgical operative exposure can be recovered from standard monocular operating-microscope images combined with microscope pose data. Methods: In a phantom-based laboratory study, two aneurysm training phantoms were imaged with a ZEISS Pentero 800 microscope integrated with Brainlab Cranial Navigation. Microscope images from the standard composite video output were stored with synchronous microscope poses. After intrinsic and extrinsic calibration, depth was estimated with the pretrained Depth Anything 3 model without task-specific fine-tuning. Fused point clouds were converted to meshes using Poisson surface reconstruction. Reconstructions were compared with reference surfaces from structured-light scanning and fine-slice CT. Results: For phantom A, representing a deeper surgical corridor, reconstruction accuracy ranged from 1.95 $\pm$ 1.70 mm to 2.33 $\pm$ 2.15 mm. For phantom B, representing a directly exposed surface, accuracy ranged from 1.02 $\pm$ 0.93 mm to 1.52 $\pm$ 1.21 mm. Larger image sets mainly improved completeness, while accuracy remained within a narrower range. Corridor analysis showed preservation of overall geometry with local deviations in incompletely reconstructed regions. Conclusions: Standard monocular microscope images combined with navigation-derived pose data can reconstruct millimeter-range 3D surfaces using a foundation-model-based pipeline. These results show technical feasibility in a controlled phantom setting and support further development toward objective quantification of operative exposure, image fusion, and characterization of working spaces for future surgical instrumentation.

Related papers

• Rendering Novel Views of MRI Using 3D Gaussian Splatting [54.6286495315908]
  We adapt 3D Gaussian Splatting to form a reconstruction starting from sparse anisotropic MRIs, and imaging planes aligned with the anatomy relevant for clinical evaluation are then sampled and rendered.<n>The novel view plane is optimal for diagnostic radiological grading of the target anatomy, whereas the original MRI is not.
  arXiv  Detail & Related papers  (2026-06-24T18:00:03Z)
• Accelerating 3D Photoacoustic Computed Tomography with End-to-End Physics-Aware Neural Operators [74.65171736966131]
  Photoacoustic computed tomography (PACT) combines optical contrast with ultrasonic resolution, achieving deep-tissue imaging beyond the optical diffusion limit.<n>Current implementations require dense transducer arrays and prolonged acquisition times, limiting clinical translation.<n>We introduce Pano, an end-to-end physics-aware model that directly learns the inverse acoustic mapping from sensor measurements to volumetric reconstructions.
  arXiv  Detail & Related papers  (2025-09-11T23:12:55Z)
• Neural Field-Based 3D Surface Reconstruction of Microstructures from Multi-Detector Signals in Scanning Electron Microscopy [7.293073530041304]
  NFH-SEM takes multi-view, multi-detector 2D SEM images as input and fuses geometric and photometric information into a continuous neural field representation.<n> NFH-SEM eliminates the manual calibration procedures through end-to-end self-calibration and automatically disentangles shadows from SEM images during training.
  arXiv  Detail & Related papers  (2025-08-05T20:00:57Z)
• Robotic Arm Platform for Multi-View Image Acquisition and 3D Reconstruction in Minimally Invasive Surgery [40.55055153469741]
  This work introduces a robotic arm platform for efficient multi-view image acquisition and precise 3D reconstruction in Minimally invasive surgery settings. We adapted a laparoscope to a robotic arm and captured ex-vivo images of several ovine organs across varying lighting conditions. We employed recently released learning-based feature matchers combined with COLMAP to produce our reconstructions.
  arXiv  Detail & Related papers  (2024-10-15T15:42:30Z)
• EndoPerfect: High-Accuracy Monocular Depth Estimation and 3D Reconstruction for Endoscopic Surgery via NeRF-Stereo Fusion [11.798218793025974]
  In endoscopic sinus surgery (ESS), intraoperative CT offers valuable intraoperative assessment but is constrained by slow deployment and radiation exposure.<n>Existing techniques often struggle to achieve the submillimeter precision required for dense reconstruction.<n>We propose an online learning approach that leverages Neural Radiance Fields (NeRF) as an intermediate representation.<n>Our method attains a point-to-point accuracy below 0.5 mm, with a demonstrated theoretical depth accuracy of 0.125 $pm$ 0.443 mm.
  arXiv  Detail & Related papers  (2024-10-05T05:26:21Z)
• Post-mastoidectomy Surface Multi-View Synthesis from a Single Microscopy Image [4.777201894011511]
  We introduce a novel pipeline that is capable of generating synthetic multi-view videos from a single CI microscope image.<n>We use a patient's pre-operative CT scan to predict the post-mastoidectomy surface using a method designed for this purpose.
  arXiv  Detail & Related papers  (2024-08-31T16:45:24Z)
• High-fidelity Endoscopic Image Synthesis by Utilizing Depth-guided Neural Surfaces [18.948630080040576]
  We introduce a novel method for colon section reconstruction by leveraging NeuS applied to endoscopic images, supplemented by a single frame of depth map. Our approach demonstrates exceptional accuracy in completely rendering colon sections, even capturing unseen portions of the surface. This breakthrough opens avenues for achieving stable and consistently scaled reconstructions, promising enhanced quality in cancer screening procedures and treatment interventions.
  arXiv  Detail & Related papers  (2024-04-20T18:06:26Z)
• Next-generation Surgical Navigation: Marker-less Multi-view 6DoF Pose Estimation of Surgical Instruments [64.59698930334012]
  We present a multi-camera capture setup consisting of static and head-mounted cameras.<n>Second, we publish a multi-view RGB-D video dataset of ex-vivo spine surgeries, captured in a surgical wet lab and a real operating theatre.<n>Third, we evaluate three state-of-the-art single-view and multi-view methods for the task of 6DoF pose estimation of surgical instruments.
  arXiv  Detail & Related papers  (2023-05-05T13:42:19Z)
• Passive superresolution imaging of incoherent objects [63.942632088208505]
  Method consists of measuring the field's spatial mode components in the image plane in the overcomplete basis of Hermite-Gaussian modes and their superpositions. Deep neural network is used to reconstruct the object from these measurements.
  arXiv  Detail & Related papers  (2023-04-19T15:53:09Z)
• Low dosage 3D volume fluorescence microscopy imaging using compressive sensing [0.0]
  We present a compressive sensing (CS) based approach to fully reconstruct 3D volumes with the same signal-to-noise ratio (SNR) with less than half of the excitation dosage. We demonstrate our technique by capturing a 3D volume of the RFP labeled neurons in the zebrafish embryo spinal cord with the axial sampling of 0.1um using a confocal microscope. The developed CS-based methodology in this work can be easily applied to other deep imaging modalities such as two-photon and light-sheet microscopy, where reducing sample photo-toxicity is a critical challenge.
  arXiv  Detail & Related papers  (2022-01-03T18:44:50Z)
• 3D Reconstruction of Curvilinear Structures with Stereo Matching DeepConvolutional Neural Networks [52.710012864395246]
  We propose a fully automated pipeline for both detection and matching of curvilinear structures in stereo pairs. We mainly focus on 3D reconstruction of dislocations from stereo pairs of TEM images.
  arXiv  Detail & Related papers  (2021-10-14T23:05:47Z)
• Appearance Learning for Image-based Motion Estimation in Tomography [60.980769164955454]
  In tomographic imaging, anatomical structures are reconstructed by applying a pseudo-inverse forward model to acquired signals. Patient motion corrupts the geometry alignment in the reconstruction process resulting in motion artifacts. We propose an appearance learning approach recognizing the structures of rigid motion independently from the scanned object.
  arXiv  Detail & Related papers  (2020-06-18T09:49:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.