Neural-Augmented Kelvinlet: Real-Time Soft Tissue Deformation with Multiple Graspers

• URL: http://arxiv.org/abs/2506.08043v1
• Date: Fri, 06 Jun 2025 19:22:49 GMT
• Title: Neural-Augmented Kelvinlet: Real-Time Soft Tissue Deformation with Multiple Graspers
• Authors: Ashkan Shahbazi, Kyvia Pereira, Jon S. Heiselman, Elaheh Akbari, Annie C. Benson, Sepehr Seifi, Xinyuan Liu, Garrison L. Johnston, Erwin Terpstra, Anne Draaisma, Jan-Jaap Severes, Jie Ying Wu, Nabil Simaan, Michael L. Miga, Soheil Kolouri,
• Abstract summary: We introduce a novel physics-informed neural simulator that approximates soft tissue deformations in a realistic and real-time manner.<n>Our framework integrates Kelvinlet-based priors into neural simulators, making it the first approach to leverage Kelvinlets for residual learning and regularization.
• Score: 11.97310697947584
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Fast and accurate simulation of soft tissue deformation is a critical factor for surgical robotics and medical training. In this paper, we introduce a novel physics-informed neural simulator that approximates soft tissue deformations in a realistic and real-time manner. Our framework integrates Kelvinlet-based priors into neural simulators, making it the first approach to leverage Kelvinlets for residual learning and regularization in data-driven soft tissue modeling. By incorporating large-scale Finite Element Method (FEM) simulations of both linear and nonlinear soft tissue responses, our method improves neural network predictions across diverse architectures, enhancing accuracy and physical consistency while maintaining low latency for real-time performance. We demonstrate the effectiveness of our approach by performing accurate surgical maneuvers that simulate the use of standard laparoscopic tissue grasping tools with high fidelity. These results establish Kelvinlet-augmented learning as a powerful and efficient strategy for real-time, physics-aware soft tissue simulation in surgical applications.

Related papers

• Conditional Graph Neural Network for Predicting Soft Tissue Deformation and Forces [0.9986418756990159]
  We introduce a novel data-driven model, a conditional graph neural network (cGNN) to tackle this complexity.<n>Our model takes surface points and the location of applied forces, and is specifically designed to predict the deformation of the points and the forces exerted on them.<n>We trained our model on experimentally collected surface tracking data of a soft tissue phantom and used transfer learning to overcome the data scarcity.
  arXiv  Detail & Related papers  (2025-07-07T13:33:39Z)
• Hybrid Neural-MPM for Interactive Fluid Simulations in Real-Time [57.30651532625017]
  We present a novel hybrid method that integrates numerical simulation, neural physics, and generative control.<n>Our system demonstrates robust performance across diverse 2D/3D scenarios, material types, and obstacle interactions.<n>We promise to release both models and data upon acceptance.
  arXiv  Detail & Related papers  (2025-05-25T01:27:18Z)
• MIXPINN: Mixed-Material Simulations by Physics-Informed Neural Network [1.275845610262865]
  Traditional Finite Element Method (FEM)-based simulations are computationally expensive and impractical for real-time scenarios.<n>We introduce MIXPINN, a physics-informed Graph Neural Network (GNN) framework for mixed-material simulations.<n>By leveraging a graph-based representation of biomechanical structures, MIXPINN learns high-fidelity deformations from FEM-generated data and achieves real-time inference with sub-millimeter accuracy.
  arXiv  Detail & Related papers  (2025-03-17T12:48:29Z)
• GausSim: Foreseeing Reality by Gaussian Simulator for Elastic Objects [55.02281855589641]
  GausSim is a novel neural network-based simulator designed to capture the dynamic behaviors of real-world elastic objects represented through Gaussian kernels.<n>We leverage continuum mechanics and treat each kernel as a Center of Mass System (CMS) that represents continuous piece of matter.<n>In addition, GausSim incorporates explicit physics constraints, such as mass and momentum conservation, ensuring interpretable results and robust, physically plausible simulations.
  arXiv  Detail & Related papers  (2024-12-23T18:58:17Z)
• Thermodynamics-informed graph neural networks for real-time simulation of digital human twins [2.6811507121199325]
  This paper presents a novel methodology aimed at advancing current lines of research in soft tissue simulation.<n>The proposed approach integrates the geometric bias of graph neural networks with the physical bias derived from the imposition of a metriplectic structure.<n>Based on the adopted methodologies, we propose a model that predicts human liver responses to traction and compression loads in as little as 7.3 milliseconds.
  arXiv  Detail & Related papers  (2024-12-16T18:01:40Z)
• Realistic Surgical Simulation from Monocular Videos [49.114370617718244]
  This paper tackles the challenge of automatically performing realistic surgical simulations from readily available surgical videos.<n>We propose SurgiSim, a novel automatic simulation system to overcome these limitations.<n> Experiments on various surgical scenarios and interactions demonstrate SurgiSim's ability to perform realistic simulation of soft tissues.
  arXiv  Detail & Related papers  (2024-12-03T10:32:41Z)
• Graph Neural Networks for modelling breast biomechanical compression [0.08192907805418582]
  Breast compression simulation is essential for accurate image registration from 3D modalities to X-ray procedures like mammography. It accounts for tissue shape and position changes due to compression, ensuring precise alignment and improved analysis. Finite Element Analysis (FEA) is reliable for approximating soft tissue deformation, it struggles with balancing accuracy and computational efficiency. Recent studies have used data-driven models trained on FEA results to speed up tissue deformation predictions. We propose to explore Physics-based Graph Neural Networks (PhysGNN) for breast compression simulation.
  arXiv  Detail & Related papers  (2024-11-10T20:59:23Z)
• A Realistic Simulation Framework for Analog/Digital Neuromorphic Architectures [73.65190161312555]
  ARCANA is a software spiking neural network simulator designed to account for the properties of mixed-signal neuromorphic circuits.<n>We show how the results obtained provide a reliable estimate of the behavior of the spiking neural network trained in software, once deployed in hardware.
  arXiv  Detail & Related papers  (2024-09-23T11:16:46Z)
• Efficient Deformable Tissue Reconstruction via Orthogonal Neural Plane [58.871015937204255]
  We introduce Fast Orthogonal Plane (plane) for the reconstruction of deformable tissues. We conceptualize surgical procedures as 4D volumes, and break them down into static and dynamic fields comprised of neural planes. This factorization iscretizes four-dimensional space, leading to a decreased memory usage and faster optimization.
  arXiv  Detail & Related papers  (2023-12-23T13:27:50Z)
• DiffuseBot: Breeding Soft Robots With Physics-Augmented Generative Diffusion Models [102.13968267347553]
  We present DiffuseBot, a physics-augmented diffusion model that generates soft robot morphologies capable of excelling in a wide spectrum of tasks. We showcase a range of simulated and fabricated robots along with their capabilities.
  arXiv  Detail & Related papers  (2023-11-28T18:58:48Z)
• Real-time simulation of viscoelastic tissue behavior with physics-guided deep learning [0.8250374560598492]
  We propose a deep learning method for predicting displacement fields of soft tissues with viselastic properties. The proposed method achieves a better accuracy over the conventional CNN models. It is hoped that the present investigation will help in filling the gap in applying deep learning in virtual reality.
  arXiv  Detail & Related papers  (2023-01-11T18:17:10Z)
• Conditional Generative Models for Simulation of EMG During Naturalistic Movements [45.698312905115955]
  We present a conditional generative neural network trained adversarially to generate motor unit activation potential waveforms. We demonstrate the ability of such a model to predictively interpolate between a much smaller number of numerical model's outputs with a high accuracy.
  arXiv  Detail & Related papers  (2022-11-03T14:49:02Z)
• PhysGNN: A Physics-Driven Graph Neural Network Based Model for Predicting Soft Tissue Deformation in Image-Guided Neurosurgery [0.15229257192293202]
  We propose a data-driven model that approximates the solution of finite element analysis (FEA) by leveraging graph neural networks (GNNs) We demonstrate that the proposed architecture, PhysGNN, promises accurate and fast soft tissue deformation approximations while remaining computationally feasible, suitable for neurosurgical settings.
  arXiv  Detail & Related papers  (2021-09-09T15:43:59Z)
• Data-driven generation of plausible tissue geometries for realistic photoacoustic image synthesis [53.65837038435433]
  Photoacoustic tomography (PAT) has the potential to recover morphological and functional tissue properties. We propose a novel approach to PAT data simulation, which we refer to as "learning to simulate" We leverage the concept of Generative Adversarial Networks (GANs) trained on semantically annotated medical imaging data to generate plausible tissue geometries.
  arXiv  Detail & Related papers  (2021-03-29T11:30:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.