ST-ColoNet: Spatio-Temporal Colon Segment Recognition via Hybrid Attention and Edge-Guided Feature Learning

• URL: http://arxiv.org/abs/2605.28119v3
• Date: Tue, 02 Jun 2026 01:54:12 GMT
• Title: ST-ColoNet: Spatio-Temporal Colon Segment Recognition via Hybrid Attention and Edge-Guided Feature Learning
• Abstract summary: Existing automatic recognition methods only use colonoscopy images without fully exploiting the use of temporal information, leading to poor performance.<n>We propose a two-stage deep learning-based framework, Colo-Segment Recognition via SpatioTemporal Network (ST-ColoNet)<n>We show that our framework is capable of achieving state-of-the-art performance on the task of colo-segment recognition, achieving an accuracy of 81.0% and F1-score of 70.7%.
• Score: 16.290114015268532
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Colo-segment recognition in colonoscopy videos is a key requirement for many downstream tasks, but existing automatic recognition methods only use colonoscopy images without fully exploiting the use of temporal information, leading to poor performance. Additionally, relevant public video-based datasets are in scarcity. To tackle this problem, we curate and release a labeled dataset specifically for the task of colo-segment recognition. In addition, we propose a two-stage deep learning-based framework, Colo-Segment Recognition via SpatioTemporal Network (ST-ColoNet), for the task of colo-segment recognition from colonoscopy videos which includes the Colorlaus module that uses metric learning to optimize edge-mediated spatial feature extraction, as well as the Full-Temp module which combines three self-attention patterns to better approximate full self-attention on long colonoscopy sequences and optimize temporal feature aggregation. Through extensive ablation experiments, we show that our framework is capable of achieving state-of-the-art performance on the task of colo-segment recognition, achieving an accuracy of 81.0% and F1-score of 70.7%, which is a tremendous improvement over state-of-the-art methods.

Related papers

• Contrastive Learning under Noisy Temporal Self-Supervision for Colonoscopy Videos [8.245519663146586]
  Learning robust representations of polyp tracklets is key to enabling AI-assisted colonoscopy applications.<n>In this work, we leverage the sequential workflow of colonoscopy procedures to derive self-supervised associations from temporal structure.<n>We demonstrate the effectiveness of the learned representations across multiple downstream tasks, including polyp retrieval and re-identification, size estimation, and histology classification.
  arXiv  Detail & Related papers  (2026-05-12T16:04:42Z)
• Training-free Temporal Object Tracking in Surgical Videos [20.689670966095097]
  We present a novel approach for online object tracking in laparoscopic cholecystectomy (LC) surgical videos.<n>Our method addresses the challenges of costly pixel-level annotations and label inconsistencies inherent in existing datasets.
  arXiv  Detail & Related papers  (2026-03-08T23:09:16Z)
• Adapting Vision Foundation Models for Real-time Ultrasound Image Segmentation [20.009670139005085]
  Existing ultrasound segmentation methods often struggle with adaptability to new tasks.<n>We introduce an adaptive framework that leverages the vision foundation model Hiera to extract multi-scale features.<n>These enriched features are then decoded to produce precise and robust segmentation.
  arXiv  Detail & Related papers  (2025-03-31T17:47:42Z)
• A Temporal Convolutional Network-Based Approach and a Benchmark Dataset for Colonoscopy Video Temporal Segmentation [3.146247125118741]
  ColonTCN is a learning-based architecture that employs custom temporal convolutional blocks to efficiently capture temporal dependencies for the temporal segmentation of colonoscopy videos.<n>ColonTCN achieves state-of-the-art performance in classification accuracy while maintaining a low parameter count when evaluated.<n>We believe that the proposed open-access benchmark and the ColonTCN approach represent a significant advancement in the temporal segmentation of colonoscopy procedures.
  arXiv  Detail & Related papers  (2025-02-05T18:21:56Z)
• Auxiliary Tasks Enhanced Dual-affinity Learning for Weakly Supervised Semantic Segmentation [79.05949524349005]
  We propose AuxSegNet+, a weakly supervised auxiliary learning framework to explore the rich information from saliency maps. We also propose a cross-task affinity learning mechanism to learn pixel-level affinities from the saliency and segmentation feature maps.
  arXiv  Detail & Related papers  (2024-03-02T10:03:21Z)
• Colo-SCRL: Self-Supervised Contrastive Representation Learning for Colonoscopic Video Retrieval [2.868043986903368]
  We construct a large-scale colonoscopic dataset named Colo-Pair for medical practice. Based on this dataset, a simple yet effective training method called Colo-SCRL is proposed for more robust representation learning. It aims to refine general knowledge from colonoscopies through masked autoencoder-based reconstruction and momentum contrast to improve retrieval performance.
  arXiv  Detail & Related papers  (2023-03-28T01:27:23Z)
• Pseudo-label Guided Cross-video Pixel Contrast for Robotic Surgical Scene Segmentation with Limited Annotations [72.15956198507281]
  We propose PGV-CL, a novel pseudo-label guided cross-video contrast learning method to boost scene segmentation. We extensively evaluate our method on a public robotic surgery dataset EndoVis18 and a public cataract dataset CaDIS.
  arXiv  Detail & Related papers  (2022-07-20T05:42:19Z)
• Exploring Intra- and Inter-Video Relation for Surgical Semantic Scene Segmentation [58.74791043631219]
  We propose a novel framework STswinCL that explores the complementary intra- and inter-video relations to boost segmentation performance. We extensively validate our approach on two public surgical video benchmarks, including EndoVis18 Challenge and CaDIS dataset. Experimental results demonstrate the promising performance of our method, which consistently exceeds previous state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-03-29T05:52:23Z)
• Temporal Memory Relation Network for Workflow Recognition from Surgical Video [53.20825496640025]
  We propose a novel end-to-end temporal memory relation network (TMNet) for relating long-range and multi-scale temporal patterns. We have extensively validated our approach on two benchmark surgical video datasets.
  arXiv  Detail & Related papers  (2021-03-30T13:20:26Z)
• Colonoscopy Polyp Detection: Domain Adaptation From Medical Report Images to Real-time Videos [76.37907640271806]
  We propose an Image-video-joint polyp detection network (Ivy-Net) to address the domain gap between colonoscopy images from historical medical reports and real-time videos. Experiments on the collected dataset demonstrate that our Ivy-Net achieves the state-of-the-art result on colonoscopy video.
  arXiv  Detail & Related papers  (2020-12-31T10:33:09Z)
• A Self-Supervised Gait Encoding Approach with Locality-Awareness for 3D Skeleton Based Person Re-Identification [65.18004601366066]
  Person re-identification (Re-ID) via gait features within 3D skeleton sequences is a newly-emerging topic with several advantages. This paper proposes a self-supervised gait encoding approach that can leverage unlabeled skeleton data to learn gait representations for person Re-ID.
  arXiv  Detail & Related papers  (2020-09-05T16:06:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.