Online camera-pose-free stereo endoscopic tissue deformation recovery with tissue-invariant vision-biomechanics consistency

• URL: http://arxiv.org/abs/2506.19388v1
• Date: Tue, 24 Jun 2025 07:32:57 GMT
• Title: Online camera-pose-free stereo endoscopic tissue deformation recovery with tissue-invariant vision-biomechanics consistency
• Authors: Jiahe Chen, Naoki Tomii, Ichiro Sakuma, Etsuko Kobayashi,
• Abstract summary: The concept of the canonical map is introduced to optimize tissue geometry and deformation in an online approach.<n>With the inputs of depth and optical flow, the method stably models tissue geometry and deformation even when the tissue is partially occluded or moving outside the field of view.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Tissue deformation recovery based on stereo endoscopic images is crucial for tool-tissue interaction analysis and benefits surgical navigation and autonomous soft tissue manipulation. Previous research suffers from the problems raised from camera motion, occlusion, large tissue deformation, lack of tissue-specific biomechanical priors, and reliance on offline processing. Unlike previous studies where the tissue geometry and deformation are represented by 3D points and displacements, the proposed method models tissue geometry as the 3D point and derivative map and tissue deformation as the 3D displacement and local deformation map. For a single surface point, 6 parameters are used to describe its rigid motion and 3 parameters for its local deformation. The method is formulated under the camera-centric setting, where all motions are regarded as the scene motion with respect to the camera. Inter-frame alignment is realized by optimizing the inter-frame deformation, making it unnecessary to estimate camera pose. The concept of the canonical map is introduced to optimize tissue geometry and deformation in an online approach. Quantitative and qualitative experiments were conducted using in vivo and ex vivo laparoscopic datasets. With the inputs of depth and optical flow, the method stably models tissue geometry and deformation even when the tissue is partially occluded or moving outside the field of view. Results show that the 3D reconstruction accuracy in the non-occluded and occluded areas reaches 0.37$\pm$0.27 mm and 0.39$\pm$0.21 mm in terms of surface distance, respectively. The method can also estimate surface strain distribution during various manipulations as an extra modality for mechanical-based analysis.

Related papers

• Tracking-Aware Deformation Field Estimation for Non-rigid 3D Reconstruction in Robotic Surgeries [15.231519282512567]
  It is still safety critical to be aware of even the least tissue deformation during instrument-tissue interactions.<n>We propose Tracking-Aware Deformation Field (TADF), a novel framework which reconstructs the 3D mesh along with the 3D tissue deformation simultaneously.
  arXiv  Detail & Related papers  (2025-03-04T12:33:17Z)
• Neural Image Unfolding: Flattening Sparse Anatomical Structures using Neural Fields [6.5082099033254135]
  Tomographic imaging reveals internal structures of 3D objects and is crucial for medical diagnoses.<n>Various organ-specific unfolding techniques exist to map their densely sampled 3D surfaces to a distortion-minimized 2D representation.<n>We deploy a neural field to fit the transformation of the anatomy of interest to a 2D overview image.
  arXiv  Detail & Related papers  (2024-11-27T14:58:49Z)
• Decaf: Monocular Deformation Capture for Face and Hand Interactions [77.75726740605748]
  This paper introduces the first method that allows tracking human hands interacting with human faces in 3D from single monocular RGB videos. We model hands as articulated objects inducing non-rigid face deformations during an active interaction. Our method relies on a new hand-face motion and interaction capture dataset with realistic face deformations acquired with a markerless multi-view camera system.
  arXiv  Detail & Related papers  (2023-09-28T17:59:51Z)
• Unsupervised 3D Pose Estimation with Non-Rigid Structure-from-Motion Modeling [83.76377808476039]
  We propose a new modeling method for human pose deformations and design an accompanying diffusion-based motion prior. Inspired by the field of non-rigid structure-from-motion, we divide the task of reconstructing 3D human skeletons in motion into the estimation of a 3D reference skeleton. A mixed spatial-temporal NRSfMformer is used to simultaneously estimate the 3D reference skeleton and the skeleton deformation of each frame from 2D observations sequence.
  arXiv  Detail & Related papers  (2023-08-18T16:41:57Z)
• Tracking monocular camera pose and deformation for SLAM inside the human body [2.094821665776961]
  We propose a novel method to simultaneously track the camera pose and the 3D scene deformation. The method uses an illumination-invariant photometric method to track image features and estimates camera motion and deformation. Our results in simulated colonoscopies show the method's accuracy and robustness in complex scenes under increasing levels of deformation.
  arXiv  Detail & Related papers  (2022-04-18T13:25:23Z)
• {\phi}-SfT: Shape-from-Template with a Physics-Based Deformation Model [69.27632025495512]
  Shape-from-Template (SfT) methods estimate 3D surface deformations from a single monocular RGB camera. This paper proposes a new SfT approach explaining 2D observations through physical simulations.
  arXiv  Detail & Related papers  (2022-03-22T17:59:57Z)
• 3D Reconstruction of Curvilinear Structures with Stereo Matching DeepConvolutional Neural Networks [52.710012864395246]
  We propose a fully automated pipeline for both detection and matching of curvilinear structures in stereo pairs. We mainly focus on 3D reconstruction of dislocations from stereo pairs of TEM images.
  arXiv  Detail & Related papers  (2021-10-14T23:05:47Z)
• Revisiting 3D Context Modeling with Supervised Pre-training for Universal Lesion Detection in CT Slices [48.85784310158493]
  We propose a Modified Pseudo-3D Feature Pyramid Network (MP3D FPN) to efficiently extract 3D context enhanced 2D features for universal lesion detection in CT slices. With the novel pre-training method, the proposed MP3D FPN achieves state-of-the-art detection performance on the DeepLesion dataset. The proposed 3D pre-trained weights can potentially be used to boost the performance of other 3D medical image analysis tasks.
  arXiv  Detail & Related papers  (2020-12-16T07:11:16Z)
• Tattoo tomography: Freehand 3D photoacoustic image reconstruction with an optical pattern [49.240017254888336]
  Photoacoustic tomography (PAT) is a novel imaging technique that can resolve both morphological and functional tissue properties. A current drawback is the limited field-of-view provided by the conventionally applied 2D probes. We present a novel approach to 3D reconstruction of PAT data that does not require an external tracking system.
  arXiv  Detail & Related papers  (2020-11-10T09:27:56Z)
• Dynamic Reconstruction of Deformable Soft-tissue with Stereo Scope in Minimal Invasive Surgery [24.411005883017832]
  In minimal invasive surgery, it is important to rebuild and visualize the latest deformed shape of soft-tissue surfaces. This paper proposes an innovative Simultaneous localization and Mapping (SLAM) algorithm for deformable dense reconstruction of surfaces. In-vivo experiments with publicly available datasets demonstrate that the 3D models can be incrementally built for different soft-tissues.
  arXiv  Detail & Related papers  (2020-03-22T16:50:38Z)
• A new geodesic-based feature for characterization of 3D shapes: application to soft tissue organ temporal deformations [0.0]
  We show a direct application on a study of organ temporal deformations. We characterize the behavior of a bladder during a forced respiratory motion with a reduced number of 3D surface points. We demonstrate the robustness of our feature on both synthetic 3D shapes and realistic dynamic MRI data.
  arXiv  Detail & Related papers  (2020-03-18T16:56:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.