Leveraging Vision and Kinematics Data to Improve Realism of Biomechanic Soft-tissue Simulation for Robotic Surgery

• URL: http://arxiv.org/abs/2003.06518v1
• Date: Sat, 14 Mar 2020 00:16:08 GMT
• Title: Leveraging Vision and Kinematics Data to Improve Realism of Biomechanic Soft-tissue Simulation for Robotic Surgery
• Authors: Jie Ying Wu, Peter Kazanzides, Mathias Unberath
• Abstract summary: We investigate how live data acquired during any robotic endoscopic surgical procedure may be used to correct for inaccurate FEM simulation results. We use an open-source da Vinci Surgical System to probe a soft-tissue phantom and replay the interaction in simulation. We train the network to correct for the difference between the predicted mesh position and the measured point cloud.
• Score: 13.657060682152409
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Purpose Surgical simulations play an increasingly important role in surgeon education and developing algorithms that enable robots to perform surgical subtasks. To model anatomy, Finite Element Method (FEM) simulations have been held as the gold standard for calculating accurate soft-tissue deformation. Unfortunately, their accuracy is highly dependent on the simulation parameters, which can be difficult to obtain. Methods In this work, we investigate how live data acquired during any robotic endoscopic surgical procedure may be used to correct for inaccurate FEM simulation results. Since FEMs are calculated from initial parameters and cannot directly incorporate observations, we propose to add a correction factor that accounts for the discrepancy between simulation and observations. We train a network to predict this correction factor. Results To evaluate our method, we use an open-source da Vinci Surgical System to probe a soft-tissue phantom and replay the interaction in simulation. We train the network to correct for the difference between the predicted mesh position and the measured point cloud. This results in 15-30% improvement in the mean distance, demonstrating the effectiveness of our approach across a large range of simulation parameters. Conclusion We show a first step towards a framework that synergistically combines the benefits of model-based simulation and real-time observations. It corrects discrepancies between simulation and the scene that results from inaccurate modeling parameters. This can provide a more accurate simulation environment for surgeons and better data with which to train algorithms.

Related papers

• Embed and Emulate: Contrastive representations for simulation-based inference [11.543221890134399]
  This paper introduces Embed and Emulate (E&E), a new simulation-based inference ( SBI) method based on contrastive learning. E&E learns a low-dimensional latent embedding of the data and a corresponding fast emulator in the latent space. We demonstrate superior performance over existing methods in a realistic, non-identifiable parameter estimation task.
  arXiv  Detail & Related papers  (2024-09-27T02:37:01Z)
• Bridging the Sim-to-Real Gap with Bayesian Inference [53.61496586090384]
  We present SIM-FSVGD for learning robot dynamics from data. We use low-fidelity physical priors to regularize the training of neural network models. We demonstrate the effectiveness of SIM-FSVGD in bridging the sim-to-real gap on a high-performance RC racecar system.
  arXiv  Detail & Related papers  (2024-03-25T11:29:32Z)
• A Multi-Grained Symmetric Differential Equation Model for Learning Protein-Ligand Binding Dynamics [74.93549765488103]
  In drug discovery, molecular dynamics simulation provides a powerful tool for predicting binding affinities, estimating transport properties, and exploring pocket sites. We propose NeuralMD, the first machine learning surrogate that can facilitate numerical MD and provide accurate simulations in protein-ligand binding. We show the efficiency and effectiveness of NeuralMD, with a 2000$times$ speedup over standard numerical MD simulation and outperforming all other ML approaches by up to 80% under the stability metric.
  arXiv  Detail & Related papers  (2024-01-26T09:35:17Z)
• Near-realtime Facial Animation by Deep 3D Simulation Super-Resolution [7.14576106770047]
  We present a neural network-based simulation framework that can efficiently and realistically enhance a facial performance produced by a low-cost, realtime physics-based simulation. We use face animation as an exemplar of such a simulation domain, where creating this semantic congruence is achieved by simply dialing in the same muscle actuation controls and skeletal pose in the two simulators. Our proposed neural network super-resolution framework generalizes from this training set to unseen expressions, compensates for modeling discrepancies between the two simulations due to limited resolution or cost-cutting approximations in the real-time variant, and does not require any semantic descriptors or parameters to
  arXiv  Detail & Related papers  (2023-05-05T00:09:24Z)
• Real-time simulation of viscoelastic tissue behavior with physics-guided deep learning [0.8250374560598492]
  We propose a deep learning method for predicting displacement fields of soft tissues with viselastic properties. The proposed method achieves a better accuracy over the conventional CNN models. It is hoped that the present investigation will help in filling the gap in applying deep learning in virtual reality.
  arXiv  Detail & Related papers  (2023-01-11T18:17:10Z)
• Neural Posterior Estimation with Differentiable Simulators [58.720142291102135]
  We present a new method to perform Neural Posterior Estimation (NPE) with a differentiable simulator. We demonstrate how gradient information helps constrain the shape of the posterior and improves sample-efficiency.
  arXiv  Detail & Related papers  (2022-07-12T16:08:04Z)
• DiSECt: A Differentiable Simulator for Parameter Inference and Control in Robotic Cutting [71.50844437057555]
  We present DiSECt: the first differentiable simulator for cutting soft materials. The simulator augments the finite element method with a continuous contact model based on signed distance fields. We show that the simulator can be calibrated to match resultant forces and fields from a state-of-the-art commercial solver.
  arXiv  Detail & Related papers  (2022-03-19T07:27:19Z)
• Prediction of progressive lens performance from neural network simulations [62.997667081978825]
  The purpose of this study is to present a framework to predict visual acuity (VA) based on a convolutional neural network (CNN) The proposed holistic simulation tool was shown to act as an accurate model for subjective visual performance.
  arXiv  Detail & Related papers  (2021-03-19T14:51:02Z)
• Cognitive simulation models for inertial confinement fusion: Combining simulation and experimental data [0.0]
  Researchers rely heavily on computer simulations to explore the design space in search of high-performing implosions. For more effective design and investigation, simulations require input from past experimental data to better predict future performance. We describe a cognitive simulation method for combining simulation and experimental data into a common, predictive model.
  arXiv  Detail & Related papers  (2021-03-19T02:00:14Z)
• Point Cloud Based Reinforcement Learning for Sim-to-Real and Partial Observability in Visual Navigation [62.22058066456076]
  Reinforcement Learning (RL) represents powerful tools to solve complex robotic tasks. RL does not work directly in the real-world, which is known as the sim-to-real transfer problem. We propose a method that learns on an observation space constructed by point clouds and environment randomization.
  arXiv  Detail & Related papers  (2020-07-27T17:46:59Z)
• Patient Specific Biomechanics Are Clinically Significant In Accurate Computer Aided Surgical Image Guidance [3.5760618920650398]
  Augmented Reality is used in Image Guided surgery (AR IG) to fuse surgical landmarks from preoperative images into a video overlay. Physical simulation is essential to maintaining accurate position of the landmarks as surgery progresses. In liver procedures, AR IG simulation accuracy is hampered by an inability to model stiffness variations to the patients disease. We introduce a novel method to account for patient specific stiffness variation based on Magnetic Resonance Elastography (MRE) data.
  arXiv  Detail & Related papers  (2020-01-29T08:11:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.