Spatiotemporal Learning with Context-aware Video Tubelets for Ultrasound Video Analysis

• URL: http://arxiv.org/abs/2503.17475v1
• Date: Fri, 21 Mar 2025 18:39:42 GMT
• Title: Spatiotemporal Learning with Context-aware Video Tubelets for Ultrasound Video Analysis
• Authors: Gary Y. Li, Li Chen, Bryson Hicks, Nikolai Schnittke, David O. Kessler, Jeffrey Shupp, Maria Parker, Cristiana Baloescu, Christopher Moore, Cynthia Gregory, Kenton Gregory, Balasundar Raju, Jochen Kruecker, Alvin Chen,
• Abstract summary: We propose a lightweight framework for tubelet-based object detection and video classifying.<n>To address the loss of global context, we embed tubelet location, size, and confidence as inputs to the classifier.<n>Our method is efficient, with the tubelet comprising only 0.4M parameters.
• Score: 4.611737599608456
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Computer-aided pathology detection algorithms for video-based imaging modalities must accurately interpret complex spatiotemporal information by integrating findings across multiple frames. Current state-of-the-art methods operate by classifying on video sub-volumes (tubelets), but they often lose global spatial context by focusing only on local regions within detection ROIs. Here we propose a lightweight framework for tubelet-based object detection and video classification that preserves both global spatial context and fine spatiotemporal features. To address the loss of global context, we embed tubelet location, size, and confidence as inputs to the classifier. Additionally, we use ROI-aligned feature maps from a pre-trained detection model, leveraging learned feature representations to increase the receptive field and reduce computational complexity. Our method is efficient, with the spatiotemporal tubelet classifier comprising only 0.4M parameters. We apply our approach to detect and classify lung consolidation and pleural effusion in ultrasound videos. Five-fold cross-validation on 14,804 videos from 828 patients shows our method outperforms previous tubelet-based approaches and is suited for real-time workflows.

Related papers

• Deepfake Detection with Spatio-Temporal Consistency and Attention [46.1135899490656]
  Deepfake videos are causing growing concerns among communities due to their ever-increasing realism.<n>Current methods for detecting forged videos rely mainly on global frame features.<n>We propose a neural Deepfake detector that focuses on the localized manipulative signatures of the forged videos.
  arXiv  Detail & Related papers  (2025-02-12T08:51:33Z)
• Weakly Supervised Video Anomaly Detection and Localization with Spatio-Temporal Prompts [57.01985221057047]
  This paper introduces a novel method that learnstemporal prompt embeddings for weakly supervised video anomaly detection and localization (WSVADL) based on pre-trained vision-language models (VLMs) Our method achieves state-of-theart performance on three public benchmarks for the WSVADL task.
  arXiv  Detail & Related papers  (2024-08-12T03:31:29Z)
• A Spatial-Temporal Deformable Attention based Framework for Breast Lesion Detection in Videos [107.96514633713034]
  We propose a spatial-temporal deformable attention based framework, named STNet. Our STNet introduces a spatial-temporal deformable attention module to perform local spatial-temporal feature fusion. Experiments on the public breast lesion ultrasound video dataset show that our STNet obtains a state-of-the-art detection performance.
  arXiv  Detail & Related papers  (2023-09-09T07:00:10Z)
• Video-SwinUNet: Spatio-temporal Deep Learning Framework for VFSS Instance Segmentation [10.789826145990016]
  This paper presents a deep learning framework for medical video segmentation. Our framework explicitly extracts features from neighbouring frames across the temporal dimension. It incorporates them with a temporal feature blender, which then tokenises the high-level-temporal feature to form a strong global feature encoded via a Swin Transformer.
  arXiv  Detail & Related papers  (2023-02-22T12:09:39Z)
• RLogist: Fast Observation Strategy on Whole-slide Images with Deep Reinforcement Learning [15.955265218706467]
  Whole-slide images (WSI) in computational pathology have high resolution with gigapixel size, but are generally with sparse regions of interest. We develop RLogist, a deep reinforcement learning (DRL) method for fast observation strategy on WSIs.
  arXiv  Detail & Related papers  (2022-12-04T04:03:34Z)
• Video-TransUNet: Temporally Blended Vision Transformer for CT VFSS Instance Segmentation [11.575821326313607]
  We propose Video-TransUNet, a deep architecture for segmentation in medical CT videos constructed by integrating temporal feature blending into the TransUNet deep learning framework. In particular, our approach amalgamates strong frame representation via a ResNet CNN backbone, multi-frame feature blending via a Temporal Context Module, and reconstructive capabilities for multiple targets via a UNet-based convolutional-deconal architecture with multiple heads.
  arXiv  Detail & Related papers  (2022-08-17T14:28:58Z)
• Exploring Intra- and Inter-Video Relation for Surgical Semantic Scene Segmentation [58.74791043631219]
  We propose a novel framework STswinCL that explores the complementary intra- and inter-video relations to boost segmentation performance. We extensively validate our approach on two public surgical video benchmarks, including EndoVis18 Challenge and CaDIS dataset. Experimental results demonstrate the promising performance of our method, which consistently exceeds previous state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-03-29T05:52:23Z)
• Video Salient Object Detection via Contrastive Features and Attention Modules [106.33219760012048]
  We propose a network with attention modules to learn contrastive features for video salient object detection. A co-attention formulation is utilized to combine the low-level and high-level features. We show that the proposed method requires less computation, and performs favorably against the state-of-the-art approaches.
  arXiv  Detail & Related papers  (2021-11-03T17:40:32Z)
• Spatial-Temporal Correlation and Topology Learning for Person Re-Identification in Videos [78.45050529204701]
  We propose a novel framework to pursue discriminative and robust representation by modeling cross-scale spatial-temporal correlation. CTL utilizes a CNN backbone and a key-points estimator to extract semantic local features from human body. It explores a context-reinforced topology to construct multi-scale graphs by considering both global contextual information and physical connections of human body.
  arXiv  Detail & Related papers  (2021-04-15T14:32:12Z)
• LRTD: Long-Range Temporal Dependency based Active Learning for Surgical Workflow Recognition [67.86810761677403]
  We propose a novel active learning method for cost-effective surgical video analysis. Specifically, we propose a non-local recurrent convolutional network (NL-RCNet), which introduces non-local block to capture the long-range temporal dependency. We validate our approach on a large surgical video dataset (Cholec80) by performing surgical workflow recognition task.
  arXiv  Detail & Related papers  (2020-04-21T09:21:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.