Ultrasound Video Summarization using Deep Reinforcement Learning

• URL: http://arxiv.org/abs/2005.09531v1
• Date: Tue, 19 May 2020 15:44:18 GMT
• Title: Ultrasound Video Summarization using Deep Reinforcement Learning
• Authors: Tianrui Liu, Qingjie Meng, Athanasios Vlontzos, Jeremy Tan, Daniel Rueckert and Bernhard Kainz
• Abstract summary: We introduce a fully automatic video summarization method tailored to the needs of medical video data. We show that our method is superior to alternative video summarization methods and that it preserves essential information required by clinical diagnostic standards.
• Score: 12.320114045092291
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Video is an essential imaging modality for diagnostics, e.g. in ultrasound imaging, for endoscopy, or movement assessment. However, video hasn't received a lot of attention in the medical image analysis community. In the clinical practice, it is challenging to utilise raw diagnostic video data efficiently as video data takes a long time to process, annotate or audit. In this paper we introduce a novel, fully automatic video summarization method that is tailored to the needs of medical video data. Our approach is framed as reinforcement learning problem and produces agents focusing on the preservation of important diagnostic information. We evaluate our method on videos from fetal ultrasound screening, where commonly only a small amount of the recorded data is used diagnostically. We show that our method is superior to alternative video summarization methods and that it preserves essential information required by clinical diagnostic standards.

Related papers

• VideoPath-LLaVA: Pathology Diagnostic Reasoning Through Video Instruction Tuning [2.6954348706500766]
  We present VideoPath-LLaVA, the first large multimodal model (LMM) in computational pathology.<n>It integrates three distinct image scenarios, single patch images, automatically-extracted clips, and manually segmented video pathology images.<n>By generating detailed histological descriptions and culminating in a definitive sign-out diagnosis, VideoPath-LLaVA bridges visual narratives with diagnostic reasoning.
  arXiv  Detail & Related papers  (2025-05-07T07:41:19Z)
• YouTube Video Analytics for Patient Engagement: Evidence from Colonoscopy Preparation Videos [3.7941428390253193]
  This study demonstrates a data analysis pipeline that utilizes methods to retrieve medical information from YouTube videos. We first use the YouTube Data API to collect metadata of desired videos on select search keywords. Then we annotate the YouTube video materials on medical information, video understandability and overall recommendation.
  arXiv  Detail & Related papers  (2024-10-01T19:38:46Z)
• Breast Ultrasound Report Generation using LangChain [58.07183284468881]
  We propose the integration of multiple image analysis tools through a LangChain using Large Language Models (LLM) into the breast reporting process. Our method can accurately extract relevant features from ultrasound images, interpret them in a clinical context, and produce comprehensive and standardized reports.
  arXiv  Detail & Related papers  (2023-12-05T00:28:26Z)
• Open video data sharing in developmental and behavioural science [1.9312167442699324]
  Video recording is a widely used method for documenting infant and child behaviours. The need of shared large-scaled datasets remains increasing. To share data while abiding by privacy protection rules, a critical question arises whether efforts at data de-identification reduce data utility.
  arXiv  Detail & Related papers  (2022-07-22T11:47:47Z)
• Weakly-supervised High-fidelity Ultrasound Video Synthesis with Feature Decoupling [13.161739586288704]
  In clinical practice, analysis and diagnosis often rely on US sequences rather than a single image to obtain dynamic anatomical information. This is challenging for novices to learn because practicing with adequate videos from patients is clinically unpractical. We propose a novel framework to synthesize high-fidelity US videos.
  arXiv  Detail & Related papers  (2022-07-01T14:53:22Z)
• A New Dataset and A Baseline Model for Breast Lesion Detection in Ultrasound Videos [43.42513012531214]
  We first collect and annotate an ultrasound video dataset (188 videos) for breast lesion detection. We propose a clip-level and video-level feature aggregated network (CVA-Net) for addressing breast lesion detection in ultrasound videos.
  arXiv  Detail & Related papers  (2022-07-01T01:37:50Z)
• Voice-assisted Image Labelling for Endoscopic Ultrasound Classification using Neural Networks [48.732863591145964]
  We propose a multi-modal convolutional neural network architecture that labels endoscopic ultrasound (EUS) images from raw verbal comments provided by a clinician during the procedure. Our results show a prediction accuracy of 76% at image level on a dataset with 5 different labels.
  arXiv  Detail & Related papers  (2021-10-12T21:22:24Z)
• Machine Learning Methods for Histopathological Image Analysis: A Review [62.14548392474976]
  Histopathological images (HIs) are the gold standard for evaluating some types of tumors for cancer diagnosis. One of the ways of accelerating such an analysis is to use computer-aided diagnosis (CAD) systems.
  arXiv  Detail & Related papers  (2021-02-07T19:12:32Z)
• LRTD: Long-Range Temporal Dependency based Active Learning for Surgical Workflow Recognition [67.86810761677403]
  We propose a novel active learning method for cost-effective surgical video analysis. Specifically, we propose a non-local recurrent convolutional network (NL-RCNet), which introduces non-local block to capture the long-range temporal dependency. We validate our approach on a large surgical video dataset (Cholec80) by performing surgical workflow recognition task.
  arXiv  Detail & Related papers  (2020-04-21T09:21:22Z)
• Confident Coreset for Active Learning in Medical Image Analysis [57.436224561482966]
  We propose a novel active learning method, confident coreset, which considers both uncertainty and distribution for effectively selecting informative samples. By comparative experiments on two medical image analysis tasks, we show that our method outperforms other active learning methods.
  arXiv  Detail & Related papers  (2020-04-05T13:46:16Z)
• Self-trained Deep Ordinal Regression for End-to-End Video Anomaly Detection [114.9714355807607]
  We show that applying self-trained deep ordinal regression to video anomaly detection overcomes two key limitations of existing methods. We devise an end-to-end trainable video anomaly detection approach that enables joint representation learning and anomaly scoring without manually labeled normal/abnormal data.
  arXiv  Detail & Related papers  (2020-03-15T08:44:55Z)
• Self-supervised Representation Learning for Ultrasound Video [18.515314344284445]
  We propose a self-supervised learning approach to learn meaningful and transferable representations from medical imaging video. We force the model to address anatomy-aware tasks with free supervision from the data itself. Experiments on fetal ultrasound video show that the proposed approach can effectively learn meaningful and strong representations.
  arXiv  Detail & Related papers  (2020-02-28T23:00:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.