Autoregressive deep learning for real-time simulation of soft tissue dynamics during virtual neurosurgery

• URL: http://arxiv.org/abs/2601.13676v1
• Date: Tue, 20 Jan 2026 07:25:32 GMT
• Title: Autoregressive deep learning for real-time simulation of soft tissue dynamics during virtual neurosurgery
• Authors: Fabian Greifeneder, Wolfgang Fenz, Benedikt Alkin, Johannes Brandstetter, Michael Giretzlehner, Philipp Moser,
• Abstract summary: Traditional numerical solvers often fall short in meeting real-time performance requirements.<n>We introduce a deep learning-based surrogate model that efficiently simulates transient brain deformation.<n>We further integrate the trained surrogate model into an interactive neurosurgical simulation environment, achieving runtimes below 10 ms per simulation step.
• Score: 19.464888859778487
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate simulation of brain deformation is a key component for developing realistic, interactive neurosurgical simulators, as complex nonlinear deformations must be captured to ensure realistic tool-tissue interactions. However, traditional numerical solvers often fall short in meeting real-time performance requirements. To overcome this, we introduce a deep learning-based surrogate model that efficiently simulates transient brain deformation caused by continuous interactions between surgical instruments and the virtual brain geometry. Building on Universal Physics Transformers, our approach operates directly on large-scale mesh data and is trained on an extensive dataset generated from nonlinear finite element simulations, covering a broad spectrum of temporal instrument-tissue interaction scenarios. To reduce the accumulation of errors in autoregressive inference, we propose a stochastic teacher forcing strategy applied during model training. Specifically, training consists of short stochastic rollouts in which the proportion of ground truth inputs is gradually decreased in favor of model-generated predictions. Our results show that the proposed surrogate model achieves accurate and efficient predictions across a range of transient brain deformation scenarios, scaling to meshes with up to 150,000 nodes. The introduced stochastic teacher forcing technique substantially improves long-term rollout stability, reducing the maximum prediction error from 6.7 mm to 3.5 mm. We further integrate the trained surrogate model into an interactive neurosurgical simulation environment, achieving runtimes below 10 ms per simulation step on consumer-grade inference hardware. Our proposed deep learning framework enables rapid, smooth and accurate biomechanical simulations of dynamic brain tissue deformation, laying the foundation for realistic surgical training environments.

Related papers

• Neural-Augmented Kelvinlet for Real-Time Soft Tissue Deformation Modeling [13.94373407381203]
  We present a novel physics-informed neural simulation framework that enables real-time prediction of soft-tissue deformations.<n>Our approach integrates Kelvinlet-based analytical priors with large-scale FEM data, capturing both linear and nonlinear tissue responses.<n>These results establish Kelvinlet-augmented learning as a principled and computationally efficient paradigm for real-time, physics-aware soft-tissue simulation in surgical AI.
  arXiv  Detail & Related papers  (2025-06-06T19:22:49Z)
• Hybrid Neural-MPM for Interactive Fluid Simulations in Real-Time [57.30651532625017]
  We present a novel hybrid method that integrates numerical simulation, neural physics, and generative control.<n>Our system demonstrates robust performance across diverse 2D/3D scenarios, material types, and obstacle interactions.<n>We promise to release both models and data upon acceptance.
  arXiv  Detail & Related papers  (2025-05-25T01:27:18Z)
• MIXPINN: Mixed-Material Simulations by Physics-Informed Neural Network [1.275845610262865]
  Traditional Finite Element Method (FEM)-based simulations are computationally expensive and impractical for real-time scenarios.<n>We introduce MIXPINN, a physics-informed Graph Neural Network (GNN) framework for mixed-material simulations.<n>By leveraging a graph-based representation of biomechanical structures, MIXPINN learns high-fidelity deformations from FEM-generated data and achieves real-time inference with sub-millimeter accuracy.
  arXiv  Detail & Related papers  (2025-03-17T12:48:29Z)
• Thermodynamics-informed graph neural networks for real-time simulation of digital human twins [2.6811507121199325]
  This paper presents a novel methodology aimed at advancing current lines of research in soft tissue simulation.<n>The proposed approach integrates the geometric bias of graph neural networks with the physical bias derived from the imposition of a metriplectic structure.<n>Based on the adopted methodologies, we propose a model that predicts human liver responses to traction and compression loads in as little as 7.3 milliseconds.
  arXiv  Detail & Related papers  (2024-12-16T18:01:40Z)
• A Multi-Grained Symmetric Differential Equation Model for Learning Protein-Ligand Binding Dynamics [73.35846234413611]
  In drug discovery, molecular dynamics (MD) simulation provides a powerful tool for predicting binding affinities, estimating transport properties, and exploring pocket sites. We propose NeuralMD, the first machine learning (ML) surrogate that can facilitate numerical MD and provide accurate simulations in protein-ligand binding dynamics. We demonstrate the efficiency and effectiveness of NeuralMD, achieving over 1K$times$ speedup compared to standard numerical MD simulations.
  arXiv  Detail & Related papers  (2024-01-26T09:35:17Z)
• Real-time simulation of viscoelastic tissue behavior with physics-guided deep learning [0.8250374560598492]
  We propose a deep learning method for predicting displacement fields of soft tissues with viselastic properties. The proposed method achieves a better accuracy over the conventional CNN models. It is hoped that the present investigation will help in filling the gap in applying deep learning in virtual reality.
  arXiv  Detail & Related papers  (2023-01-11T18:17:10Z)
• Continual learning autoencoder training for a particle-in-cell simulation via streaming [52.77024349608834]
  upcoming exascale era will provide a new generation of physics simulations with high resolution. These simulations will have a high resolution, which will impact the training of machine learning models since storing a high amount of simulation data on disk is nearly impossible. This work presents an approach that trains a neural network concurrently to a running simulation without data on a disk.
  arXiv  Detail & Related papers  (2022-11-09T09:55:14Z)
• Conditional Generative Models for Simulation of EMG During Naturalistic Movements [45.698312905115955]
  We present a conditional generative neural network trained adversarially to generate motor unit activation potential waveforms. We demonstrate the ability of such a model to predictively interpolate between a much smaller number of numerical model's outputs with a high accuracy.
  arXiv  Detail & Related papers  (2022-11-03T14:49:02Z)
• Real-to-Sim: Predicting Residual Errors of Robotic Systems with Sparse Data using a Learning-based Unscented Kalman Filter [65.93205328894608]
  We learn the residual errors between a dynamic and/or simulator model and the real robot. We show that with the learned residual errors, we can further close the reality gap between dynamic models, simulations, and actual hardware.
  arXiv  Detail & Related papers  (2022-09-07T15:15:12Z)
• Multi-fidelity Hierarchical Neural Processes [79.0284780825048]
  Multi-fidelity surrogate modeling reduces the computational cost by fusing different simulation outputs. We propose Multi-fidelity Hierarchical Neural Processes (MF-HNP), a unified neural latent variable model for multi-fidelity surrogate modeling. We evaluate MF-HNP on epidemiology and climate modeling tasks, achieving competitive performance in terms of accuracy and uncertainty estimation.
  arXiv  Detail & Related papers  (2022-06-10T04:54:13Z)
• What Robot do I Need? Fast Co-Adaptation of Morphology and Control using Graph Neural Networks [7.261920381796185]
  A major challenge for the application of co-adaptation methods to the real world is the simulation-to-reality-gap. This paper presents a new approach combining classic high-frequency deep neural networks with computational expensive Graph Neural Networks for the data-efficient co-adaptation of agents.
  arXiv  Detail & Related papers  (2021-11-03T17:41:38Z)
• Deep Bayesian Active Learning for Accelerating Stochastic Simulation [74.58219903138301]
  Interactive Neural Process (INP) is a deep active learning framework for simulations and with active learning approaches. For active learning, we propose a novel acquisition function, Latent Information Gain (LIG), calculated in the latent space of NP based models. The results demonstrate STNP outperforms the baselines in the learning setting and LIG achieves the state-of-the-art for active learning.
  arXiv  Detail & Related papers  (2021-06-05T01:31:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.