Dynamic Reconstruction of Deformable Soft-tissue with Stereo Scope in Minimal Invasive Surgery

• URL: http://arxiv.org/abs/2003.10867v1
• Date: Sun, 22 Mar 2020 16:50:38 GMT
• Title: Dynamic Reconstruction of Deformable Soft-tissue with Stereo Scope in Minimal Invasive Surgery
• Authors: Jingwei Song, Jun Wang, Liang Zhao, Shoudong Huang and Gamini Dissanayake
• Abstract summary: In minimal invasive surgery, it is important to rebuild and visualize the latest deformed shape of soft-tissue surfaces. This paper proposes an innovative Simultaneous localization and Mapping (SLAM) algorithm for deformable dense reconstruction of surfaces. In-vivo experiments with publicly available datasets demonstrate that the 3D models can be incrementally built for different soft-tissues.
• Score: 24.411005883017832
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In minimal invasive surgery, it is important to rebuild and visualize the latest deformed shape of soft-tissue surfaces to mitigate tissue damages. This paper proposes an innovative Simultaneous Localization and Mapping (SLAM) algorithm for deformable dense reconstruction of surfaces using a sequence of images from a stereoscope. We introduce a warping field based on the Embedded Deformation (ED) nodes with 3D shapes recovered from consecutive pairs of stereo images. The warping field is estimated by deforming the last updated model to the current live model. Our SLAM system can: (1) Incrementally build a live model by progressively fusing new observations with vivid accurate texture. (2) Estimate the deformed shape of unobserved region with the principle As-Rigid-As-Possible. (3) Show the consecutive shape of models. (4) Estimate the current relative pose between the soft-tissue and the scope. In-vivo experiments with publicly available datasets demonstrate that the 3D models can be incrementally built for different soft-tissues with different deformations from sequences of stereo images obtained by laparoscopes. Results show the potential clinical application of our SLAM system for providing surgeon useful shape and texture information in minimal invasive surgery.

Related papers

• High-fidelity Endoscopic Image Synthesis by Utilizing Depth-guided Neural Surfaces [18.948630080040576]
  We introduce a novel method for colon section reconstruction by leveraging NeuS applied to endoscopic images, supplemented by a single frame of depth map. Our approach demonstrates exceptional accuracy in completely rendering colon sections, even capturing unseen portions of the surface. This breakthrough opens avenues for achieving stable and consistently scaled reconstructions, promising enhanced quality in cancer screening procedures and treatment interventions.
  arXiv  Detail & Related papers  (2024-04-20T18:06:26Z)
• FLex: Joint Pose and Dynamic Radiance Fields Optimization for Stereo Endoscopic Videos [79.50191812646125]
  Reconstruction of endoscopic scenes is an important asset for various medical applications, from post-surgery analysis to educational training. We adress the challenging setup of a moving endoscope within a highly dynamic environment of deforming tissue. We propose an implicit scene separation into multiple overlapping 4D neural radiance fields (NeRFs) and a progressive optimization scheme jointly optimizing for reconstruction and camera poses from scratch. This improves the ease-of-use and allows to scale reconstruction capabilities in time to process surgical videos of 5,000 frames and more; an improvement of more than ten times compared to the state of the art while being agnostic to external tracking information
  arXiv  Detail & Related papers  (2024-03-18T19:13:02Z)
• Implicit Shape Modeling for Anatomical Structure Refinement of Volumetric Medical Images [29.894934602946567]
  We propose a unified framework for 3D shape modelling and segmentation refinement based on implicit neural networks. For improved shape representation, implicit shape constraints are used for both instances and latent templates. Experiments on validation datasets involving liver, pancreas and lung segmentation demonstrate the superiority of our approach.
  arXiv  Detail & Related papers  (2023-12-11T07:09:32Z)
• On the Localization of Ultrasound Image Slices within Point Distribution Models [84.27083443424408]
  Thyroid disorders are most commonly diagnosed using high-resolution Ultrasound (US) Longitudinal tracking is a pivotal diagnostic protocol for monitoring changes in pathological thyroid morphology. We present a framework for automated US image slice localization within a 3D shape representation.
  arXiv  Detail & Related papers  (2023-09-01T10:10:46Z)
• EndoSurf: Neural Surface Reconstruction of Deformable Tissues with Stereo Endoscope Videos [72.59573904930419]
  Reconstructing soft tissues from stereo endoscope videos is an essential prerequisite for many medical applications. Previous methods struggle to produce high-quality geometry and appearance due to their inadequate representations of 3D scenes. We propose a novel neural-field-based method, called EndoSurf, which effectively learns to represent a deforming surface from an RGBD sequence.
  arXiv  Detail & Related papers  (2023-07-21T02:28:20Z)
• CNN-based real-time 2D-3D deformable registration from a single X-ray projection [2.1198879079315573]
  This paper presents a method for real-time 2D-3D non-rigid registration using a single fluoroscopic image. A dataset composed of displacement fields and 2D projections of the anatomy is generated from a preoperative scan. A neural network is trained to recover the unknown 3D displacement field from a single projection image.
  arXiv  Detail & Related papers  (2022-12-15T09:57:19Z)
• Unbiased 4D: Monocular 4D Reconstruction with a Neural Deformation Model [76.64071133839862]
  Capturing general deforming scenes from monocular RGB video is crucial for many computer graphics and vision applications. Our method, Ub4D, handles large deformations, performs shape completion in occluded regions, and can operate on monocular RGB videos directly by using differentiable volume rendering. Results on our new dataset, which will be made publicly available, demonstrate a clear improvement over the state of the art in terms of surface reconstruction accuracy and robustness to large deformations.
  arXiv  Detail & Related papers  (2022-06-16T17:59:54Z)
• IGCN: Image-to-graph Convolutional Network for 2D/3D Deformable Registration [1.2246649738388387]
  We propose an image-to-graph convolutional network that achieves deformable registration of a 3D organ mesh for a single-viewpoint 2D projection image. We show shape prediction considering relationships among multiple organs can be used to predict respiratory motion and deformation from radiographs with clinically acceptable accuracy.
  arXiv  Detail & Related papers  (2021-10-31T12:48:37Z)
• 3D Reconstruction of Curvilinear Structures with Stereo Matching DeepConvolutional Neural Networks [52.710012864395246]
  We propose a fully automated pipeline for both detection and matching of curvilinear structures in stereo pairs. We mainly focus on 3D reconstruction of dislocations from stereo pairs of TEM images.
  arXiv  Detail & Related papers  (2021-10-14T23:05:47Z)
• Tattoo tomography: Freehand 3D photoacoustic image reconstruction with an optical pattern [49.240017254888336]
  Photoacoustic tomography (PAT) is a novel imaging technique that can resolve both morphological and functional tissue properties. A current drawback is the limited field-of-view provided by the conventionally applied 2D probes. We present a novel approach to 3D reconstruction of PAT data that does not require an external tracking system.
  arXiv  Detail & Related papers  (2020-11-10T09:27:56Z)
• A new geodesic-based feature for characterization of 3D shapes: application to soft tissue organ temporal deformations [0.0]
  We show a direct application on a study of organ temporal deformations. We characterize the behavior of a bladder during a forced respiratory motion with a reduced number of 3D surface points. We demonstrate the robustness of our feature on both synthetic 3D shapes and realistic dynamic MRI data.
  arXiv  Detail & Related papers  (2020-03-18T16:56:41Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.