Surgical Video Generation From Diffusion to World Models: A Survey

• URL: http://arxiv.org/abs/2608.26214v1
• Date: Wed, 26 Aug 2026 08:27:20 GMT
• Title: Surgical Video Generation From Diffusion to World Models: A Survey
• Abstract summary: This survey organizes the 2024-2026 literature into three categories: unconditional generation, conditional generation, and world modeling generation.<n>We examine the persistent gap between pixel-level fidelity and clinical plausibility, and identify generalization, physical realism, controllability, and interpretability as bottlenecks.<n>This survey provides a structured overview of the current state and open challenges, offering a reference for researchers working at the intersection of intelligent perception, multi-modal fusion, generative AI, and surgical data science.
• Score: 21.02261559352406
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Surgical video data provides the primary training resource for models of intraoperative perception, surgical workflow understanding, and robotic decision-making. However, clinical data acquisition remains constrained by privacy, cost, and class imbalance. Surgical video generation has emerged as a transformative approach to addressing data scarcity and as a foundation for surgical simulation, training, and robotic policy learning. The field has developed rapidly without a clear conceptual framework. This survey organizes the 2024-2026 literature into three categories: unconditional generation, conditional generation, and world modeling generation, revealing a fundamental shift in how the task is defined from synthesizing visually plausible frames to modeling the causal dynamics of surgical scenes. We examine the persistent gap between pixel-level fidelity and clinical plausibility, and identify generalization, physical realism, controllability, and interpretability as bottlenecks. We further summarize experimental results of representative methods on public datasets to provide a quantitative reference for the field. This survey provides a structured overview of the current state and open challenges, offering a reference for researchers working at the intersection of intelligent perception, multi-modal fusion, generative AI, and surgical data science.

Related papers

• Towards World Models in Biomedical Research [77.59710966764466]
  We propose biomedical world models as a paradigm for AI-driven discovery.<n>These models learn latent representations of molecular, cellular, tissue and clinical states, together with intervention-conditioned dynamics.<n>We discuss how biomedical world models could function as data engines, environment simulators and scientific planning substrates across applications including virtual cells, organoids, virtual patients and surgical simulation.
  arXiv  Detail & Related papers  (2026-06-04T09:28:54Z)
• A Systematic Review on Data-Driven Brain Deformation Modeling for Image-Guided Neurosurgery [29.978282112659993]
  Accurate compensation of brain deformation is a critical challenge for reliable image-guided neurosurgery.<n>Recent AI-driven approaches developed between January 2020 and April 2025 for modeling and correcting brain deformation.
  arXiv  Detail & Related papers  (2026-02-09T22:11:56Z)
• How Far Are Surgeons from Surgical World Models? A Pilot Study on Zero-shot Surgical Video Generation with Expert Assessment [69.13598421861654]
  We present SurgVeo, the first expert-curated benchmark for video generation model evaluation in surgery.<n>We task the advanced Veo-3 model with a zero-shot prediction task on surgical clips from laparoscopic and neurosurgical procedures.<n>Our results reveal a distinct "plausibility gap": while Veo-3 achieves exceptional Visual Perceptual Plausibility, it fails critically at higher levels of the Surgical Plausibility Pyramid.
  arXiv  Detail & Related papers  (2025-11-03T17:28:54Z)
• Decoding the Surgical Scene: A Scoping Review of Scene Graphs in Surgery [36.192962258966105]
  Scene graphs (SGs) provide structured representations crucial for decoding complex, dynamic surgical environments.<n>This PRISMA-ScR-guided scoping review systematically maps the evolving landscape of SG research in surgery.<n>Our analysis reveals rapid growth, yet uncovers a critical 'data divide'<n>SGs are maturing into an essential semantic bridge, enabling a new generation of intelligent systems to improve surgical safety, efficiency, and training.
  arXiv  Detail & Related papers  (2025-09-25T09:25:46Z)
• Data-Efficient Learning for Generalizable Surgical Video Understanding [0.0]
  This research aims to bridge gap between deep learning-based surgical video analysis in research and its real-world clinical environments.<n>I benchmarked state-of-the-art neural network architectures to identify the most effective designs for each task.<n>We developed semi-driven frameworks that improve model performance across tasks by leveraging large amounts of unlabeled surgical video.
  arXiv  Detail & Related papers  (2025-08-13T22:00:23Z)
• Content Generation Models in Computational Pathology: A Comprehensive Survey on Methods, Applications, and Challenges [55.41538606569424]
  Review focuses on four key domains: image generation, text generation, molecular profile-morphology generation, and other specialized generation applications.<n>We trace the evolution of content generation architectures -- from early generative adversarial networks to recent advances in diffusion models and generative vision-language models.<n>Review concludes with a discussion of open challenges and prospective research directions, with an emphasis on developing integrated and clinically deployable generation systems.
  arXiv  Detail & Related papers  (2025-05-16T08:44:50Z)
• Monocular pose estimation of articulated surgical instruments in open surgery [0.873811641236639]
  This work presents a novel approach to monocular 6D pose estimation of surgical instruments in open surgery, addressing challenges such as object articulations, symmetries, and lack of annotated real-world data. The proposed approach consists of three main components: (1) synthetic data generation using 3D modeling of surgical tools with articulation rigging; (2) a tailored pose estimation framework combining object detection with pose estimation and a hybrid geometric fusion strategy; and (3) a training strategy that utilizes both synthetic and real unannotated data, employing domain adaptation on real video data using automatically generated pseudo-labels.
  arXiv  Detail & Related papers  (2024-07-16T19:47:35Z)
• MindBridge: A Cross-Subject Brain Decoding Framework [60.58552697067837]
  Brain decoding aims to reconstruct stimuli from acquired brain signals. Currently, brain decoding is confined to a per-subject-per-model paradigm. We present MindBridge, that achieves cross-subject brain decoding by employing only one model.
  arXiv  Detail & Related papers  (2024-04-11T15:46:42Z)
• Multimodal Semantic Scene Graphs for Holistic Modeling of Surgical Procedures [70.69948035469467]
  We take advantage of the latest computer vision methodologies for generating 3D graphs from camera views. We then introduce the Multimodal Semantic Graph Scene (MSSG) which aims at providing unified symbolic and semantic representation of surgical procedures.
  arXiv  Detail & Related papers  (2021-06-09T14:35:44Z)
• Surgical Visual Domain Adaptation: Results from the MICCAI 2020 SurgVisDom Challenge [9.986124942784969]
  This work seeks to explore the potential for visual domain adaptation in surgery to overcome data privacy concerns. In particular, we propose to use video from virtual reality (VR) simulations of surgical exercises to develop algorithms to recognize tasks in a clinical-like setting. We present the performance of the different approaches to solve visual domain adaptation developed by challenge participants.
  arXiv  Detail & Related papers  (2021-02-26T18:45:28Z)
• Relational Graph Learning on Visual and Kinematics Embeddings for Accurate Gesture Recognition in Robotic Surgery [84.73764603474413]
  We propose a novel online approach of multi-modal graph network (i.e., MRG-Net) to dynamically integrate visual and kinematics information. The effectiveness of our method is demonstrated with state-of-the-art results on the public JIGSAWS dataset.
  arXiv  Detail & Related papers  (2020-11-03T11:00:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.