SurgeMOD: Translating image-space tissue motions into vision-based surgical forces

• URL: http://arxiv.org/abs/2406.17707v1
• Date: Tue, 25 Jun 2024 16:46:21 GMT
• Title: SurgeMOD: Translating image-space tissue motions into vision-based surgical forces
• Authors: Mikel De Iturrate Reyzabal, Dionysios Malas, Shuai Wang, Sebastien Ourselin, Hongbin Liu,
• Abstract summary: We present a new approach for vision-based force estimation in Minimally Invasive Robotic Surgery. Using internal movements generated by natural processes like breathing or the cardiac cycle, we infer the image-space basis of the motion on the frequency domain. We demonstrate that this method can estimate point contact forces reliably for silicone phantom and ex-vivo experiments.
• Score: 6.4474263352749075
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present a new approach for vision-based force estimation in Minimally Invasive Robotic Surgery based on frequency domain basis of motion of organs derived directly from video. Using internal movements generated by natural processes like breathing or the cardiac cycle, we infer the image-space basis of the motion on the frequency domain. As we are working with this representation, we discretize the problem to a limited amount of low-frequencies to build an image-space mechanical model of the environment. We use this pre-built model to define our force estimation problem as a dynamic constraint problem. We demonstrate that this method can estimate point contact forces reliably for silicone phantom and ex-vivo experiments, matching real readings from a force sensor. In addition, we perform qualitative experiments in which we synthesize coherent force textures from surgical videos over a certain region of interest selected by the user. Our method demonstrates good results for both quantitative and qualitative analysis, providing a good starting point for a purely vision-based method for surgical force estimation.

Related papers

• Gear-NeRF: Free-Viewpoint Rendering and Tracking with Motion-aware Spatio-Temporal Sampling [70.34875558830241]
  We present a way for learning a-temporal (4D) embedding, based on semantic semantic gears to allow for stratified modeling of dynamic regions of rendering the scene. At the same time, almost for free, our tracking approach enables free-viewpoint of interest - a functionality not yet achieved by existing NeRF-based methods.
  arXiv  Detail & Related papers  (2024-06-06T03:37:39Z)
• MS-MANO: Enabling Hand Pose Tracking with Biomechanical Constraints [50.61346764110482]
  We integrate a musculoskeletal system with a learnable parametric hand model, MANO, to create MS-MANO. This model emulates the dynamics of muscles and tendons to drive the skeletal system, imposing physiologically realistic constraints on the resulting torque trajectories. We also propose a simulation-in-the-loop pose refinement framework, BioPR, that refines the initial estimated pose through a multi-layer perceptron network.
  arXiv  Detail & Related papers  (2024-04-16T02:18:18Z)
• Implicit Neural Representation for Physics-driven Actuated Soft Bodies [15.261578025057593]
  This paper utilizes a differentiable, quasi-static, and physics-based simulation layer to optimize for actuation signals parameterized by neural networks. We define a function that enables a continuous mapping from a spatial point in the material space to the actuation value. We extend our implicit model to mandible kinematics for the particular case of facial animation and show that we can reliably reproduce facial expressions captured with high-quality capture systems.
  arXiv  Detail & Related papers  (2024-01-26T13:42:12Z)
• Diffusion Priors for Dynamic View Synthesis from Monocular Videos [59.42406064983643]
  Dynamic novel view synthesis aims to capture the temporal evolution of visual content within videos. We first finetune a pretrained RGB-D diffusion model on the video frames using a customization technique. We distill the knowledge from the finetuned model to a 4D representations encompassing both dynamic and static Neural Radiance Fields.
  arXiv  Detail & Related papers  (2024-01-10T23:26:41Z)
• SISMIK for brain MRI: Deep-learning-based motion estimation and model-based motion correction in k-space [0.0]
  We propose a retrospective method for motion estimation and correction for 2D Spin-Echo scans of the brain. The method leverages the power of deep neural networks to estimate motion parameters in k-space. It uses a model-based approach to restore degraded images to avoid ''hallucinations''
  arXiv  Detail & Related papers  (2023-12-20T17:38:56Z)
• Modelling Human Visual Motion Processing with Trainable Motion Energy Sensing and a Self-attention Network [1.9458156037869137]
  We propose an image-computable model of human motion perception by bridging the gap between biological and computer vision models. This model architecture aims to capture the computations in V1-MT, the core structure for motion perception in the biological visual system. In silico neurophysiology reveals that our model's unit responses are similar to mammalian neural recordings regarding motion pooling and speed tuning.
  arXiv  Detail & Related papers  (2023-05-16T04:16:07Z)
• Learning How To Robustly Estimate Camera Pose in Endoscopic Videos [5.073761189475753]
  We propose a solution for stereo endoscopes that estimates depth and optical flow to minimize two geometric losses for camera pose estimation. Most importantly, we introduce two learned adaptive per-pixel weight mappings that balance contributions according to the input image content. We validate our approach on the publicly available SCARED dataset and introduce a new in-vivo dataset, StereoMIS.
  arXiv  Detail & Related papers  (2023-04-17T07:05:01Z)
• Differentiable Frequency-based Disentanglement for Aerial Video Action Recognition [56.91538445510214]
  We present a learning algorithm for human activity recognition in videos. Our approach is designed for UAV videos, which are mainly acquired from obliquely placed dynamic cameras. We conduct extensive experiments on the UAV Human dataset and the NEC Drone dataset.
  arXiv  Detail & Related papers  (2022-09-15T22:16:52Z)
• Contact-Aware Retargeting of Skinned Motion [49.71236739408685]
  This paper introduces a motion estimation method that preserves self-contacts and prevents interpenetration. The method identifies self-contacts and ground contacts in the input motion, and optimize the motion to apply to the output skeleton. In experiments, our results quantitatively outperform previous methods and we conduct a user study where our retargeted motions are rated as higher-quality than those produced by recent works.
  arXiv  Detail & Related papers  (2021-09-15T17:05:02Z)
• Optical Flow Estimation from a Single Motion-blurred Image [66.2061278123057]
  Motion blur in an image may have practical interests in fundamental computer vision problems. We propose a novel framework to estimate optical flow from a single motion-blurred image in an end-to-end manner.
  arXiv  Detail & Related papers  (2021-03-04T12:45:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.