Action-grounded tissue affordance enables anticipatory auto-framing that lowers surgeon cognitive workload during laparoscopic surgery

• URL: http://arxiv.org/abs/2608.02471v1
• Date: Mon, 03 Aug 2026 16:38:47 GMT
• Title: Action-grounded tissue affordance enables anticipatory auto-framing that lowers surgeon cognitive workload during laparoscopic surgery
• Abstract summary: We introduce DiffeoAfford, an action-grounded tissue affordance framework that retrospectively derives visual attention supervision from completed surgical procedures.<n>A real-time prediction model trained on affordance hotspot labels anticipates relevant surgical regions and enables AffordView, an assistive auto-framing system for laparoscopic visualization.
• Score: 16.980132212521536
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Computational attention models could help surgeons manage the visual demands of laparoscopy, but they require dense spatial labels that are difficult to obtain because surgical intent is highly specialized and tacit. Here, we introduce DiffeoAfford, an action-grounded tissue affordance framework that retrospectively derives visual attention supervision from completed surgical procedures. By combining diffeomorphism-constrained tissue tracking with instrument trajectory analysis, DiffeoAfford generates affordance hotspot labels without manual per-frame annotation. A real-time prediction model trained on these labels anticipates relevant surgical regions and enables AffordView, an assistive auto-framing system for laparoscopic visualization. The proposed framework aligns with expert annotations and intraoperative surgeon gaze, and reduces surgeon cognitive workload during real-world evaluations using subjective, physiological, and behavioral measures.

Related papers

• Beyond Instrument Motion: Recognizing Tissue Tension Toward Surgical Skill Assessment [36.50919068544186]
  We introduce tissue tension recognition, a new clinically motivated video understanding task for laparoscopic and robot-assisted rectal cancer surgery.<n>To support this task, we construct SurgTension, the first expert-annotated tissue tension dataset.<n>We also propose TensionTRAC, a lightweight trajectory-based framework that models tissue tension from sparse point trajectories.
  arXiv  Detail & Related papers  (2026-08-18T15:56:08Z)
• First Investigation of Deep Learning for Intraoperative Gauze Segmentation in Minimally Invasive Abdominal Surgery [3.258646009285695]
  We investigate the use of deep learning methods for gauze segmentation in robot-assisted minimally invasive abdominal surgeries.<n>The training data reflects realistic surgical settings and captures extensive diversity in spatial, morphological, and visual attributes.<n>We evaluate several widely used segmentation architectures to establish a proof-of-concept for gauze segmentation in a realistic clinical setting.
  arXiv  Detail & Related papers  (2026-07-31T08:03:47Z)
• SurgAM: Surgical Affordance Map Prediction with Multimodal Feature Fusion for Robot Autonomy [24.007662271891277]
  We introduce surgical affordance prediction, which identifies actionable regions for fundamental surgical actions from visual data.<n>A novel adaptive feature fusion framework is proposed that leverages the complementary strengths of a self-supervised vision transformer encoder.<n>A scene-guided attention decoder is proposed to focus on critical surgical areas while suppressing background distractions.
  arXiv  Detail & Related papers  (2026-07-05T16:12:01Z)
• Attachment Anchors: A Novel Framework for Laparoscopic Grasping Point Prediction in Colorectal Surgery [19.147229560255]
  We introduce attachment anchors, a structured representation that encodes the local geometric and mechanical relationships between tissue and its anatomical attachments in colorectal surgery.<n>This representation reduces uncertainty in grasping point prediction by normalizing surgical scenes into a consistent local reference frame.<n>Experiments on a dataset of 90 colorectal surgeries demonstrate that attachment anchors improve grasping point prediction compared to image-only baselines.
  arXiv  Detail & Related papers  (2026-02-19T12:19:56Z)
• Semantic Segmentation for Preoperative Planning in Transcatheter Aortic Valve Replacement [61.573750959726475]
  We consider medical guidelines for preoperative planning of the transcatheter aortic valve replacement (TAVR) and identify tasks that may be supported via semantic segmentation models.<n>We first derive fine-grained TAVR-relevant pseudo-labels from coarse-grained anatomical information, in order to train segmentation models and quantify how well they are able to find these structures in the scans.
  arXiv  Detail & Related papers  (2025-07-22T13:24:45Z)
• Benchmarking Laparoscopic Surgical Image Restoration and Beyond [54.28852320829451]
  In laparoscopic surgery, a clear and high-quality visual field is critical for surgeons to make accurate decisions.<n> persistent visual degradation, including smoke generated by energy devices, lens fogging from thermal gradients, and lens contamination pose risks to patient safety.<n>We introduce a real-world open-source surgical image restoration dataset covering laparoscopic environments, called SurgClean.
  arXiv  Detail & Related papers  (2025-05-25T14:17:56Z)
• Surgical Foundation Model Leveraging Compression and Entropy Maximization for Image-Guided Surgical Assistance [50.486523249499115]
  Real-time video understanding is critical to guide procedures in minimally invasive surgery (MIS)<n>We propose Compress-to-Explore (C2E), a novel self-supervised framework to learn compact, informative representations from surgical videos.<n>C2E uses entropy-maximizing decoders to compress images while preserving clinically relevant details, improving encoder performance without labeled data.
  arXiv  Detail & Related papers  (2025-05-16T14:02:24Z)
• Surgeons vs. Computer Vision: A comparative analysis on surgical phase recognition capabilities [65.66373425605278]
  Automated Surgical Phase Recognition (SPR) uses Artificial Intelligence (AI) to segment the surgical workflow into its key events.<n>Previous research has focused on short and linear surgical procedures and has not explored if temporal context influences experts' ability to better classify surgical phases.<n>This research addresses these gaps, focusing on Robot-Assisted Partial Nephrectomy (RAPN) as a highly non-linear procedure.
  arXiv  Detail & Related papers  (2025-04-26T15:37:22Z)
• SURGIVID: Annotation-Efficient Surgical Video Object Discovery [42.16556256395392]
  We propose an annotation-efficient framework for the semantic segmentation of surgical scenes. We employ image-based self-supervised object discovery to identify the most salient tools and anatomical structures in surgical videos. Our unsupervised setup reinforced with only 36 annotation labels indicates comparable localization performance with fully-supervised segmentation models.
  arXiv  Detail & Related papers  (2024-09-12T07:12:20Z)
• Real-time guidewire tracking and segmentation in intraoperative x-ray [52.51797358201872]
  We propose a two-stage deep learning framework for real-time guidewire segmentation and tracking. In the first stage, a Yolov5 detector is trained, using the original X-ray images as well as synthetic ones, to output the bounding boxes of possible target guidewires. In the second stage, a novel and efficient network is proposed to segment the guidewire in each detected bounding box.
  arXiv  Detail & Related papers  (2024-04-12T20:39:19Z)
• Hypergraph-Transformer (HGT) for Interactive Event Prediction in Laparoscopic and Robotic Surgery [47.47211257890948]
  We propose a predictive neural network that is capable of understanding and predicting critical interactive aspects of surgical workflow from intra-abdominal video.<n>We verify our approach on established surgical datasets and applications, including the detection and prediction of action triplets.<n>Our results demonstrate the superiority of our approach compared to unstructured alternatives.
  arXiv  Detail & Related papers  (2024-02-03T00:58:05Z)
• Action Recognition in Video Recordings from Gynecologic Laparoscopy [4.002010889177872]
  Action recognition is a prerequisite for many applications in laparoscopic video analysis. In this study, we design and evaluate a CNN-RNN architecture as well as a customized training-inference framework.
  arXiv  Detail & Related papers  (2023-11-30T16:15:46Z)
• Towards Unsupervised Learning for Instrument Segmentation in Robotic Surgery with Cycle-Consistent Adversarial Networks [54.00217496410142]
  We propose an unpaired image-to-image translation where the goal is to learn the mapping between an input endoscopic image and a corresponding annotation. Our approach allows to train image segmentation models without the need to acquire expensive annotations. We test our proposed method on Endovis 2017 challenge dataset and show that it is competitive with supervised segmentation methods.
  arXiv  Detail & Related papers  (2020-07-09T01:39:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.