NeRFscopy: Neural Radiance Fields for in-vivo Time-Varying Tissues from Endoscopy

• URL: http://arxiv.org/abs/2602.15775v1
• Date: Tue, 17 Feb 2026 18:05:23 GMT
• Title: NeRFscopy: Neural Radiance Fields for in-vivo Time-Varying Tissues from Endoscopy
• Authors: Laura Salort-Benejam, Antonio Agudo,
• Abstract summary: We introduce NeRFscopy, a self-supervised pipeline for novel view synthesis and 3D reconstruction of deformable endoscopic tissues from a monocular video.<n>NeRFscopy achieves accurate results in terms of novel view synthesis, outperforming competing methods across various challenging endoscopy scenes.
• Score: 18.633528239379483
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Endoscopy is essential in medical imaging, used for diagnosis, prognosis and treatment. Developing a robust dynamic 3D reconstruction pipeline for endoscopic videos could enhance visualization, improve diagnostic accuracy, aid in treatment planning, and guide surgery procedures. However, challenges arise due to the deformable nature of the tissues, the use of monocular cameras, illumination changes, occlusions and unknown camera trajectories. Inspired by neural rendering, we introduce NeRFscopy, a self-supervised pipeline for novel view synthesis and 3D reconstruction of deformable endoscopic tissues from a monocular video. NeRFscopy includes a deformable model with a canonical radiance field and a time-dependent deformation field parameterized by SE(3) transformations. In addition, the color images are efficiently exploited by introducing sophisticated terms to learn a 3D implicit model without assuming any template or pre-trained model, solely from data. NeRFscopy achieves accurate results in terms of novel view synthesis, outperforming competing methods across various challenging endoscopy scenes.

Related papers

• Accelerating 3D Photoacoustic Computed Tomography with End-to-End Physics-Aware Neural Operators [74.65171736966131]
  Photoacoustic computed tomography (PACT) combines optical contrast with ultrasonic resolution, achieving deep-tissue imaging beyond the optical diffusion limit.<n>Current implementations require dense transducer arrays and prolonged acquisition times, limiting clinical translation.<n>We introduce Pano, an end-to-end physics-aware model that directly learns the inverse acoustic mapping from sensor measurements to volumetric reconstructions.
  arXiv  Detail & Related papers  (2025-09-11T23:12:55Z)
• PR-ENDO: Physically Based Relightable Gaussian Splatting for Endoscopy [1.28795255913358]
  3D reconstruction and novel-view synthesis from endoscopic images are promising tools for enhancing diagnosis.<n>We present PR-ENDO, a novel 3D reconstruction framework leveraging the unique property of endoscopic imaging, where a single light source is closely aligned with the camera.<n> PR-ENDO achieves superior reconstruction quality compared to baseline methods on both public and in-house datasets.
  arXiv  Detail & Related papers  (2024-11-19T13:52:30Z)
• Intraoperative Registration by Cross-Modal Inverse Neural Rendering [61.687068931599846]
  We present a novel approach for 3D/2D intraoperative registration during neurosurgery via cross-modal inverse neural rendering. Our approach separates implicit neural representation into two components, handling anatomical structure preoperatively and appearance intraoperatively. We tested our method on retrospective patients' data from clinical cases, showing that our method outperforms state-of-the-art while meeting current clinical standards for registration.
  arXiv  Detail & Related papers  (2024-09-18T13:40:59Z)
• Creating a Digital Twin of Spinal Surgery: A Proof of Concept [68.37190859183663]
  Surgery digitalization is the process of creating a virtual replica of real-world surgery. We present a proof of concept (PoC) for surgery digitalization that is applied to an ex-vivo spinal surgery. We employ five RGB-D cameras for dynamic 3D reconstruction of the surgeon, a high-end camera for 3D reconstruction of the anatomy, an infrared stereo camera for surgical instrument tracking, and a laser scanner for 3D reconstruction of the operating room and data fusion.
  arXiv  Detail & Related papers  (2024-03-25T13:09:40Z)
• FLex: Joint Pose and Dynamic Radiance Fields Optimization for Stereo Endoscopic Videos [79.50191812646125]
  Reconstruction of endoscopic scenes is an important asset for various medical applications, from post-surgery analysis to educational training. We adress the challenging setup of a moving endoscope within a highly dynamic environment of deforming tissue. We propose an implicit scene separation into multiple overlapping 4D neural radiance fields (NeRFs) and a progressive optimization scheme jointly optimizing for reconstruction and camera poses from scratch. This improves the ease-of-use and allows to scale reconstruction capabilities in time to process surgical videos of 5,000 frames and more; an improvement of more than ten times compared to the state of the art while being agnostic to external tracking information
  arXiv  Detail & Related papers  (2024-03-18T19:13:02Z)
• Domain adaptation strategies for 3D reconstruction of the lumbar spine using real fluoroscopy data [9.21828361691977]
  This study tackles key obstacles in adopting surgical navigation in orthopedic surgeries. It shows an approach for generating 3D anatomical models of the spine from only a few fluoroscopic images. It achieved an 84% F1 score, matching the accuracy of our previous synthetic data-based research.
  arXiv  Detail & Related papers  (2024-01-29T10:22:45Z)
• 3D Guidewire Shape Reconstruction from Monoplane Fluoroscopic Images [7.0968125126570625]
  We propose a new method to reconstruct the 3D guidewire by utilizing CathSim, a state-of-the-art endovascular simulator. Our 3D-FGRN delivers results on par with conventional triangulation from simulated monoplane fluoroscopic images.
  arXiv  Detail & Related papers  (2023-11-19T03:20:42Z)
• Towards Abdominal 3-D Scene Rendering from Laparoscopy Surgical Videos using NeRFs [0.7106122418396085]
  We present a comprehensive examination of NeRFs in the context of laparoscopy surgical videos. NeRFs have recently gained attention thanks to their ability to generate photorealistic images from a 3-D static scene. Although our experimental results are promising, the proposed approach encounters substantial challenges.
  arXiv  Detail & Related papers  (2023-10-18T01:06:19Z)
• AiAReSeg: Catheter Detection and Segmentation in Interventional Ultrasound using Transformers [75.20925220246689]
  endovascular surgeries are performed using the golden standard of Fluoroscopy, which uses ionising radiation to visualise catheters and vasculature. This work proposes a solution using an adaptation of a state-of-the-art machine learning transformer architecture to detect and segment catheters in axial interventional Ultrasound image sequences.
  arXiv  Detail & Related papers  (2023-09-25T19:34:12Z)
• The interpretation of endobronchial ultrasound image using 3D convolutional neural network for differentiating malignant and benign mediastinal lesions [3.0969191504482247]
  The purpose of this study is to differentiate malignant and benign lesions by using endobronchial ultrasound (EBUS) image. Our model is robust to noise and able to fuse various imaging features and aspiration of EBUS videos.
  arXiv  Detail & Related papers  (2021-07-29T08:38:17Z)
• Tattoo tomography: Freehand 3D photoacoustic image reconstruction with an optical pattern [49.240017254888336]
  Photoacoustic tomography (PAT) is a novel imaging technique that can resolve both morphological and functional tissue properties. A current drawback is the limited field-of-view provided by the conventionally applied 2D probes. We present a novel approach to 3D reconstruction of PAT data that does not require an external tracking system.
  arXiv  Detail & Related papers  (2020-11-10T09:27:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.