TASL-Net: Tri-Attention Selective Learning Network for Intelligent Diagnosis of Bimodal Ultrasound Video

• URL: http://arxiv.org/abs/2409.01557v1
• Date: Tue, 3 Sep 2024 02:50:37 GMT
• Title: TASL-Net: Tri-Attention Selective Learning Network for Intelligent Diagnosis of Bimodal Ultrasound Video
• Authors: Chengqian Zhao, Zhao Yao, Zhaoyu Hu, Yuanxin Xie, Yafang Zhang, Yuanyuan Wang, Shuo Li, Jianhua Zhou, Jianqiao Zhou, Yin Wang, Jinhua Yu,
• Abstract summary: This paper proposes a novel Tri-Attention Selective Learning Network (TASL-Net) to tackle this challenge. TASL-Net embeds three types of diagnostic attention of sonographers into a mutual transformer framework for intelligent diagnosis of bimodal ultrasound videos. We conduct a detailed experimental validation of TASL-Net's performance on three datasets, including lung, breast, and liver.
• Score: 10.087796410298061
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In the intelligent diagnosis of bimodal (gray-scale and contrast-enhanced) ultrasound videos, medical domain knowledge such as the way sonographers browse videos, the particular areas they emphasize, and the features they pay special attention to, plays a decisive role in facilitating precise diagnosis. Embedding medical knowledge into the deep learning network can not only enhance performance but also boost clinical confidence and reliability of the network. However, it is an intractable challenge to automatically focus on these person- and disease-specific features in videos and to enable networks to encode bimodal information comprehensively and efficiently. This paper proposes a novel Tri-Attention Selective Learning Network (TASL-Net) to tackle this challenge and automatically embed three types of diagnostic attention of sonographers into a mutual transformer framework for intelligent diagnosis of bimodal ultrasound videos. Firstly, a time-intensity-curve-based video selector is designed to mimic the temporal attention of sonographers, thus removing a large amount of redundant information while improving computational efficiency of TASL-Net. Then, to introduce the spatial attention of the sonographers for contrast-enhanced video analysis, we propose the earliest-enhanced position detector based on structural similarity variation, on which the TASL-Net is made to focus on the differences of perfusion variation inside and outside the lesion. Finally, by proposing a mutual encoding strategy that combines convolution and transformer, TASL-Net possesses bimodal attention to structure features on gray-scale videos and to perfusion variations on contrast-enhanced videos. These modules work collaboratively and contribute to superior performance. We conduct a detailed experimental validation of TASL-Net's performance on three datasets, including lung, breast, and liver.

Related papers

• OnUVS: Online Feature Decoupling Framework for High-Fidelity Ultrasound Video Synthesis [34.07625938756013]
  Sonographers must observe corresponding dynamic anatomic structures to gather comprehensive information. The synthesis of US videos may represent a promising solution to this issue. We present a novel online feature-decoupling framework called OnUVS for high-fidelity US video synthesis.
  arXiv  Detail & Related papers  (2023-08-16T10:16:50Z)
• Inflated 3D Convolution-Transformer for Weakly-supervised Carotid Stenosis Grading with Ultrasound Videos [12.780908780402516]
  We present the first video classification framework for automatic carotid stenosis grading (CSG) We propose a novel and effective video classification network for weakly-supervised CSG. Our approach is extensively validated on a large clinically collected carotid US video dataset.
  arXiv  Detail & Related papers  (2023-06-05T02:50:06Z)
• Focused Decoding Enables 3D Anatomical Detection by Transformers [64.36530874341666]
  We propose a novel Detection Transformer for 3D anatomical structure detection, dubbed Focused Decoder. Focused Decoder leverages information from an anatomical region atlas to simultaneously deploy query anchors and restrict the cross-attention's field of view. We evaluate our proposed approach on two publicly available CT datasets and demonstrate that Focused Decoder not only provides strong detection results and thus alleviates the need for a vast amount of annotated data but also exhibits exceptional and highly intuitive explainability of results via attention weights.
  arXiv  Detail & Related papers  (2022-07-21T22:17:21Z)
• MDMMT-2: Multidomain Multimodal Transformer for Video Retrieval, One More Step Towards Generalization [65.09758931804478]
  Three different data sources are combined: weakly-supervised videos, crowd-labeled text-image pairs and text-video pairs. A careful analysis of available pre-trained networks helps to choose the best prior-knowledge ones.
  arXiv  Detail & Related papers  (2022-03-14T13:15:09Z)
• 2021 BEETL Competition: Advancing Transfer Learning for Subject Independence & Heterogenous EEG Data Sets [89.84774119537087]
  We design two transfer learning challenges around diagnostics and Brain-Computer-Interfacing (BCI) Task 1 is centred on medical diagnostics, addressing automatic sleep stage annotation across subjects. Task 2 is centred on Brain-Computer Interfacing (BCI), addressing motor imagery decoding across both subjects and data sets.
  arXiv  Detail & Related papers  (2022-02-14T12:12:20Z)
• Voice-assisted Image Labelling for Endoscopic Ultrasound Classification using Neural Networks [48.732863591145964]
  We propose a multi-modal convolutional neural network architecture that labels endoscopic ultrasound (EUS) images from raw verbal comments provided by a clinician during the procedure. Our results show a prediction accuracy of 76% at image level on a dataset with 5 different labels.
  arXiv  Detail & Related papers  (2021-10-12T21:22:24Z)
• Unsupervised multi-latent space reinforcement learning framework for video summarization in ultrasound imaging [0.0]
  The COVID-19 pandemic has highlighted the need for a tool to speed up triage in ultrasound scans. The proposed video-summarization technique is a step in this direction. We propose a new unsupervised reinforcement learning framework with novel rewards.
  arXiv  Detail & Related papers  (2021-09-03T04:50:35Z)
• Cascaded Robust Learning at Imperfect Labels for Chest X-ray Segmentation [61.09321488002978]
  We present a novel cascaded robust learning framework for chest X-ray segmentation with imperfect annotation. Our model consists of three independent network, which can effectively learn useful information from the peer networks. Our methods could achieve a significant improvement on the accuracy in segmentation tasks compared to the previous methods.
  arXiv  Detail & Related papers  (2021-04-05T15:50:16Z)
• Contextual Information Enhanced Convolutional Neural Networks for Retinal Vessel Segmentation in Color Fundus Images [0.0]
  An automatic retinal vessel segmentation system can effectively facilitate clinical diagnosis and ophthalmological research. A deep learning based method has been proposed and several customized modules have been integrated into the well-known encoder-decoder architecture U-net. As a result, the proposed method outperforms the work of predecessors and achieves state-of-the-art performance in Sensitivity/Recall, F1-score and MCC.
  arXiv  Detail & Related papers  (2021-03-25T06:10:47Z)
• Relational Graph Learning on Visual and Kinematics Embeddings for Accurate Gesture Recognition in Robotic Surgery [84.73764603474413]
  We propose a novel online approach of multi-modal graph network (i.e., MRG-Net) to dynamically integrate visual and kinematics information. The effectiveness of our method is demonstrated with state-of-the-art results on the public JIGSAWS dataset.
  arXiv  Detail & Related papers  (2020-11-03T11:00:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.