Mitigating Surgical Data Imbalance with Dual-Prediction Video Diffusion Model

• URL: http://arxiv.org/abs/2510.07345v1
• Date: Tue, 07 Oct 2025 20:29:27 GMT
• Title: Mitigating Surgical Data Imbalance with Dual-Prediction Video Diffusion Model
• Authors: Danush Kumar Venkatesh, Adam Schmidt, Muhammad Abdullah Jamal, Omid Mohareri,
• Abstract summary: $SurgiFlowVid$ is a sparse and controllable video diffusion framework for generating surgical videos of under-represented classes.<n>We validate our approach on three surgical datasets across tasks including action recognition, tool presence detection, and laparoscope motion prediction.
• Score: 7.746379804154433
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Surgical video datasets are essential for scene understanding, enabling procedural modeling and intra-operative support. However, these datasets are often heavily imbalanced, with rare actions and tools under-represented, which limits the robustness of downstream models. We address this challenge with $SurgiFlowVid$, a sparse and controllable video diffusion framework for generating surgical videos of under-represented classes. Our approach introduces a dual-prediction diffusion module that jointly denoises RGB frames and optical flow, providing temporal inductive biases to improve motion modeling from limited samples. In addition, a sparse visual encoder conditions the generation process on lightweight signals (e.g., sparse segmentation masks or RGB frames), enabling controllability without dense annotations. We validate our approach on three surgical datasets across tasks including action recognition, tool presence detection, and laparoscope motion prediction. Synthetic data generated by our method yields consistent gains of 10-20% over competitive baselines, establishing $SurgiFlowVid$ as a promising strategy to mitigate data imbalance and advance surgical video understanding methods.

Related papers

• Steering and Rectifying Latent Representation Manifolds in Frozen Multi-modal LLMs for Video Anomaly Detection [52.5174167737992]
  Video anomaly detection (VAD) aims to identify abnormal events in videos.<n>We propose SteerVAD, which advances MLLM-based VAD by shifting from passively reading to actively steering and rectifying internal representations.<n>Our method achieves state-of-the-art performance among tuning-free approaches requiring only 1% of training data.
  arXiv  Detail & Related papers  (2026-02-27T13:48:50Z)
• CtrlVDiff: Controllable Video Generation via Unified Multimodal Video Diffusion [62.04833878126661]
  We tackle the dual challenges of video understanding and controllable video generation within a unified diffusion framework.<n>We propose CtrlVDiff, a unified diffusion model trained with a Hybrid Modality Control Strategy (HMCS) that routes and fuses features from depth, normals, segmentation, edges, and graphics-based intrinsics (albedo, roughness, metallic)<n>Across understanding and generation benchmarks, CtrlVDiff delivers superior controllability and fidelity, enabling layer-wise edits (relighting, material adjustment, object insertion) and surpassing state-of-the-art baselines while remaining robust when some modalities are unavailable.
  arXiv  Detail & Related papers  (2025-11-26T07:27:11Z)
• Towards Application Aligned Synthetic Surgical Image Synthesis [3.1373284090264857]
  We introduce emphSurgical Application-Aligned Diffusion (SAADi), a new framework that aligns diffusion models with samples preferred by downstream models.<n>Our method constructs pairs of emphpreferred and emphnon-preferred synthetic images and employs lightweight fine-tuning of diffusion models to align the image generation process with downstream objectives explicitly.
  arXiv  Detail & Related papers  (2025-09-23T08:40:40Z)
• Holistic Surgical Phase Recognition with Hierarchical Input Dependent State Space Models [56.2236083600999]
  We propose a novel hierarchical input-dependent state space model for surgical video analysis.<n>Our framework incorporates a temporally consistent visual feature extractor, which appends a state space model head to a visual feature extractor to propagate temporal information.<n> Experiments have shown that our method outperforms the current state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2025-06-26T14:43:57Z)
• Mission Balance: Generating Under-represented Class Samples using Video Diffusion Models [1.5678321653327674]
  We propose a two-stage, text-based method to generate high-fidelity surgical videos for under-represented classes.<n>We evaluate our method on two downstream tasks--action recognition and intra-operative event prediction-demonstrating.
  arXiv  Detail & Related papers  (2025-05-14T23:43:29Z)
• AccVideo: Accelerating Video Diffusion Model with Synthetic Dataset [55.82208863521353]
  We propose AccVideo to reduce the inference steps for accelerating video diffusion models with synthetic dataset.<n>Our model achieves 8.5x improvements in generation speed compared to the teacher model.<n>Compared to previous accelerating methods, our approach is capable of generating videos with higher quality and resolution.
  arXiv  Detail & Related papers  (2025-03-25T08:52:07Z)
• Temporal-Consistent Video Restoration with Pre-trained Diffusion Models [51.47188802535954]
  Video restoration (VR) aims to recover high-quality videos from degraded ones.<n>Recent zero-shot VR methods using pre-trained diffusion models (DMs) suffer from approximation errors during reverse diffusion and insufficient temporal consistency.<n>We present a novel a Posterior Maximum (MAP) framework that directly parameterizes video frames in the seed space of DMs, eliminating approximation errors.
  arXiv  Detail & Related papers  (2025-03-19T03:41:56Z)
• One-Step Diffusion Model for Image Motion-Deblurring [85.76149042561507]
  We propose a one-step diffusion model for deblurring (OSDD), a novel framework that reduces the denoising process to a single step.<n>To tackle fidelity loss in diffusion models, we introduce an enhanced variational autoencoder (eVAE), which improves structural restoration.<n>Our method achieves strong performance on both full and no-reference metrics.
  arXiv  Detail & Related papers  (2025-03-09T09:39:57Z)
• SurgSora: Object-Aware Diffusion Model for Controllable Surgical Video Generation [25.963369099780113]
  SurgSora is a framework that generates high-fidelity, motion-controllable surgical videos from a single input frame and user-specified motion cues.<n>By conditioning these enriched features within the Stable Video Diffusion, SurgSora achieves state-of-the-art visual authenticity and controllability.
  arXiv  Detail & Related papers  (2024-12-18T16:34:51Z)
• Motion-Boundary-Driven Unsupervised Surgical Instrument Segmentation in Low-Quality Optical Flow [42.75298102809838]
  Unsupervised video-based surgical instrument segmentation has the potential to accelerate the adoption of robot-assisted procedures.<n>The generally low quality of optical flow in endoscopic footage poses a great challenge for unsupervised methods that rely heavily on motion cues.<n>We propose a novel approach that pinpoints motion boundaries, regions with abrupt flow changes, while selectively discarding frames with globally low-quality flow.
  arXiv  Detail & Related papers  (2024-03-15T06:19:02Z)
• Unsupervised Video Anomaly Detection with Diffusion Models Conditioned on Compact Motion Representations [17.816344808780965]
  unsupervised video anomaly detection (VAD) problem involves classifying each frame in a video as normal or abnormal, without any access to labels. To accomplish this, proposed method employs conditional diffusion models, where the input data is features extracted from pre-trained network. Our method utilizes a data-driven threshold and considers a high reconstruction error as an indicator of anomalous events.
  arXiv  Detail & Related papers  (2023-07-04T07:36:48Z)
• GSB: Group Superposition Binarization for Vision Transformer with Limited Training Samples [46.025105938192624]
  Vision Transformer (ViT) has performed remarkably in various computer vision tasks. ViT usually suffers from serious overfitting problems with a relatively limited number of training samples. We propose a novel model binarization technique, called Group Superposition Binarization (GSB)
  arXiv  Detail & Related papers  (2023-05-13T14:48:09Z)
• Fast Unsupervised Brain Anomaly Detection and Segmentation with Diffusion Models [1.6352599467675781]
  We propose a method based on diffusion models to detect and segment anomalies in brain imaging. Our diffusion models achieve competitive performance compared with autoregressive approaches across a series of experiments with 2D CT and MRI data.
  arXiv  Detail & Related papers  (2022-06-07T17:30:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.