From Spatial Semantics to Temporal Context: Leveraging Gaze Trajectory for Weakly Supervised Medical Image Segmentation

• URL: http://arxiv.org/abs/2607.26542v1
• Date: Wed, 29 Jul 2026 07:13:31 GMT
• Title: From Spatial Semantics to Temporal Context: Leveraging Gaze Trajectory for Weakly Supervised Medical Image Segmentation
• Abstract summary: Eye tracking offers a cost-effective solution that can be naturally integrated into clinical.<n>We propose Trajectory-guided Uncertainty-aware Network (TrailNet)<n>TrailNet exploits gaze-supervised medical image segmentation from spatial semantics to temporal context by jointly leveraging fixations and trajectories.
• Score: 5.809537365593592
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Medical image segmentation heavily depends on labor-intensive and time-consuming pixel-level annotations. Eye tracking offers a cost-effective solution that can be naturally integrated into clinical workflows. Recorded by eye trackers, gaze conveys the spatial regions of clinicians' attention through fixations and the temporal context of clinicians' progressive visual perception from trajectories. Nevertheless, effective modeling of temporal trajectories remains challenging, and noise in gaze caused by exploratory fixations greatly limits segmentation performance. To overcome these limitations, we propose the Trajectory-guided Uncertainty-aware Network (TrailNet), which exploits gaze-supervised medical image segmentation from spatial semantics modeling to temporal context by jointly leveraging fixations and trajectories. Specifically, the proposed trajectory-guided spatio-temporal encoder models temporal context and establishes complementary interactions with image spatial semantics to strengthen target perception. Furthermore, the multi-scale uncertainty decoder leverages category mutual-exclusivity constraints to produce deterministic predictions and mitigate supervision uncertainty induced by noise. To enable gaze-free inference, we further introduce a cycle distillation strategy that transfers feature-level knowledge via teacher-student networks. Experimental results on two public datasets demonstrate that TrailNet outperforms state-of-the-art methods, achieving Dice scores of 81.25% and 81.85%, respectively.

Related papers

• Latent-to-Latent Flow for Volumetric Stochastic Segmentation [42.44847226668341]
  Flow matching has emerged as a powerful framework for generative modelling.<n>We introduce a latent-to-latent flow technique for segmentation of medical volumes via encoded representations of both the image and label space.
  arXiv  Detail & Related papers  (2026-09-07T13:10:57Z)
• Towards Voxel Spacing Consistency for Medical Image Segmentation [56.992050146446495]
  We study the correlation between image resampling and segmentation, and propose Consispace, a semantic-aware resampling framework.<n> Consispace achieves consistent voxel spacing in the axial direction while preserving anatomical and semantic consistency.
  arXiv  Detail & Related papers  (2026-06-30T15:42:23Z)
• Temporally Consistent Label Interpolation for Robust Surgical Multi-Task Learning under Challenging Conditions [4.361526134899725]
  Flow-guided temporal for Robust Operating Scenes (FAROS)<n>A flow-guided label framework that combines zero-shot segmentation-based mask propagation with optical flow estimation to overcome the limitations of appearance-based propagation under challenging surgical conditions.<n>A unified Transformer-based multi-task framework that jointly learns surgical phase recognition, step recognition, anticipation, instrument segmentation, and action recognition.
  arXiv  Detail & Related papers  (2026-06-25T05:55:26Z)
• Rethinking Vector Field Learning for Generative Segmentation [50.08025820235397]
  Taming diffusion models for generative segmentation has attracted increasing attention.<n>We revisit diffusion segmentation from the perspective of vector field learning.<n>We propose a principled vector field reshaping strategy that augments the learned vanishing velocity field with a detached distance-aware correction term.<n>This correction introduces both attractive and repulsive interactions, enhancing gradient magnitudes near centroids while preserving the original diffusion training framework.
  arXiv  Detail & Related papers  (2026-03-19T17:58:19Z)
• SutureAgent: Learning Surgical Trajectories via Goal-conditioned Offline RL in Pixel Space [22.300952176139948]
  Predicting needle trajectories from endoscopic video is critical for robot-assisted suturing.<n>We formulate image-based needle trajectory prediction as a sequential decision-making problem.<n>SutureAgent encodes variable-length clips using an observation encoder to capture both local spatial cues and long-range temporal dynamics.
  arXiv  Detail & Related papers  (2026-03-19T01:36:07Z)
• Training-free Temporal Object Tracking in Surgical Videos [20.689670966095097]
  We present a novel approach for online object tracking in laparoscopic cholecystectomy (LC) surgical videos.<n>Our method addresses the challenges of costly pixel-level annotations and label inconsistencies inherent in existing datasets.
  arXiv  Detail & Related papers  (2026-03-08T23:09:16Z)
• Real-time guidewire tracking and segmentation in intraoperative x-ray [52.51797358201872]
  We propose a two-stage deep learning framework for real-time guidewire segmentation and tracking. In the first stage, a Yolov5 detector is trained, using the original X-ray images as well as synthetic ones, to output the bounding boxes of possible target guidewires. In the second stage, a novel and efficient network is proposed to segment the guidewire in each detected bounding box.
  arXiv  Detail & Related papers  (2024-04-12T20:39:19Z)
• RCPS: Rectified Contrastive Pseudo Supervision for Semi-Supervised Medical Image Segmentation [26.933651788004475]
  We propose a novel semi-supervised segmentation method named Rectified Contrastive Pseudo Supervision (RCPS) RCPS combines a rectified pseudo supervision and voxel-level contrastive learning to improve the effectiveness of semi-supervised segmentation. Experimental results reveal that the proposed method yields better segmentation performance compared with the state-of-the-art methods in semi-supervised medical image segmentation.
  arXiv  Detail & Related papers  (2023-01-13T12:03:58Z)
• Continuity-Discrimination Convolutional Neural Network for Visual Object Tracking [150.51667609413312]
  This paper proposes a novel model, named Continuity-Discrimination Convolutional Neural Network (CD-CNN) for visual object tracking. To address this problem, CD-CNN models temporal appearance continuity based on the idea of temporal slowness. In order to alleviate inaccurate target localization and drifting, we propose a novel notion, object-centroid.
  arXiv  Detail & Related papers  (2021-04-18T06:35:03Z)
• Few-shot Medical Image Segmentation using a Global Correlation Network with Discriminative Embedding [60.89561661441736]
  We propose a novel method for few-shot medical image segmentation. We construct our few-shot image segmentor using a deep convolutional network trained episodically. We enhance discriminability of deep embedding to encourage clustering of the feature domains of the same class.
  arXiv  Detail & Related papers  (2020-12-10T04:01:07Z)
• Proactive Pseudo-Intervention: Causally Informed Contrastive Learning For Interpretable Vision Models [103.64435911083432]
  We present a novel contrastive learning strategy called it Proactive Pseudo-Intervention (PPI) PPI leverages proactive interventions to guard against image features with no causal relevance. We also devise a novel causally informed salience mapping module to identify key image pixels to intervene, and show it greatly facilitates model interpretability.
  arXiv  Detail & Related papers  (2020-12-06T20:30:26Z)
• Domain aware medical image classifier interpretation by counterfactual impact analysis [2.512212190779389]
  We introduce a neural-network based attribution method, applicable to any trained predictor. Our solution identifies salient regions of an input image in a single forward-pass by measuring the effect of local image-perturbations on a predictor's score.
  arXiv  Detail & Related papers  (2020-07-13T11:11:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.