Video Polyp Segmentation: A Deep Learning Perspective

• URL: http://arxiv.org/abs/2203.14291v1
• Date: Sun, 27 Mar 2022 12:40:10 GMT
• Title: Video Polyp Segmentation: A Deep Learning Perspective
• Abstract summary: We present the first comprehensive video polyp segmentation (VPS) study. We first introduce a high-quality per-frame annotated VPS dataset, named SUN-SEG, which includes 158,690 frames from the famous SUN dataset. We also design a simple but efficient baseline, dubbed PNS+, consisting of a global encoder, a local encoder, and normalized self-attention blocks.
• Score: 97.70996418522748
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the deep learning era, we present the first comprehensive video polyp segmentation (VPS) study. Over the years, developments in VPS are not moving forward with ease due to the lack of large-scale fine-grained segmentation annotations. To tackle this issue, we first introduce a high-quality per-frame annotated VPS dataset, named SUN-SEG, which includes 158,690 frames from the famous SUN dataset. We provide additional annotations with diverse types, i.e., attribute, object mask, boundary, scribble, and polygon. Second, we design a simple but efficient baseline, dubbed PNS+, consisting of a global encoder, a local encoder, and normalized self-attention (NS) blocks. The global and local encoders receive an anchor frame and multiple successive frames to extract long-term and short-term feature representations, which are then progressively updated by two NS blocks. Extensive experiments show that PNS+ achieves the best performance and real-time inference speed (170fps), making it a promising solution for the VPS task. Third, we extensively evaluate 13 representative polyp/object segmentation models on our SUN-SEG dataset and provide attribute-based comparisons. Benchmark results are available at https: //github.com/GewelsJI/VPS.

Related papers

• First-frame Supervised Video Polyp Segmentation via Propagative and Semantic Dual-teacher Network [15.213681514145346]
  Video polyp segmentation plays a critical role in gastrointestinal cancer screening. For the first time, we reduce the annotation cost to just a single frame per polyp video, regardless of the video's length. PSDNet employs two distinct types of teachers: the propagative teacher and the semantic teacher.
  arXiv  Detail & Related papers  (2024-12-21T06:36:25Z)
• Spatio-temporal Prompting Network for Robust Video Feature Extraction [74.54597668310707]
  Frametemporal is one of the main challenges in the field of video understanding. Recent approaches exploit transformer-based integration modules to obtain quality-of-temporal information. We present a neat and unified framework called N-Temporal Prompting Network (NNSTP) It can efficiently extract video features by adjusting the input features in the network backbone.
  arXiv  Detail & Related papers  (2024-02-04T17:52:04Z)
• DVIS++: Improved Decoupled Framework for Universal Video Segmentation [30.703276476607545]
  We present OV-DVIS++, the first open-vocabulary universal video segmentation framework. By integrating CLIP with DVIS++, we present OV-DVIS++, the first open-vocabulary universal video segmentation framework.
  arXiv  Detail & Related papers  (2023-12-20T03:01:33Z)
• Lesion-aware Dynamic Kernel for Polyp Segmentation [49.63274623103663]
  We propose a lesion-aware dynamic network (LDNet) for polyp segmentation. It is a traditional u-shape encoder-decoder structure incorporated with a dynamic kernel generation and updating scheme. This simple but effective scheme endows our model with powerful segmentation performance and generalization capability.
  arXiv  Detail & Related papers  (2023-01-12T09:53:57Z)
• Progressively Normalized Self-Attention Network for Video Polyp Segmentation [88.53757948857196]
  We propose the novel PNS-Net (Progressively Normalized Self-attention Network) to efficiently learn representations from polyp videos. Our PNS-Net is based solely on a basic normalized self-attention block, equipping with recurrence and CNNs entirely. Experiments on challenging datasets demonstrate that the proposed PNS-Net achieves state-of-the-art performance.
  arXiv  Detail & Related papers  (2021-05-18T12:20:00Z)
• Reducing the Annotation Effort for Video Object Segmentation Datasets [50.893073670389164]
  densely labeling every frame with pixel masks does not scale to large datasets. We use a deep convolutional network to automatically create pseudo-labels on a pixel level from much cheaper bounding box annotations. We obtain the new TAO-VOS benchmark, which we make publicly available at www.vision.rwth-aachen.de/page/taovos.
  arXiv  Detail & Related papers  (2020-11-02T17:34:45Z)
• Directional Deep Embedding and Appearance Learning for Fast Video Object Segmentation [11.10636117512819]
  We propose a directional deep embedding and YouTube appearance learning (DEmbed) method, which is free of the online fine-tuning process. Our method achieves a state-of-the-art VOS performance without using online fine-tuning.
  arXiv  Detail & Related papers  (2020-02-17T01:51:57Z)
• Zero-Shot Video Object Segmentation via Attentive Graph Neural Networks [150.5425122989146]
  This work proposes a novel attentive graph neural network (AGNN) for zero-shot video object segmentation (ZVOS) AGNN builds a fully connected graph to efficiently represent frames as nodes, and relations between arbitrary frame pairs as edges. Experimental results on three video segmentation datasets show that AGNN sets a new state-of-the-art in each case.
  arXiv  Detail & Related papers  (2020-01-19T10:45:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.