A survey on attention mechanisms for medical applications: are we moving towards better algorithms?

• URL: http://arxiv.org/abs/2204.12406v1
• Date: Tue, 26 Apr 2022 16:04:19 GMT
• Title: A survey on attention mechanisms for medical applications: are we moving towards better algorithms?
• Authors: Tiago Gon\c{c}alves, Isabel Rio-Torto, Lu\'is F. Teixeira, Jaime S. Cardoso
• Abstract summary: This paper extensively reviews the use of attention mechanisms in machine learning for several medical applications. It proposes a critical analysis of the claims and potentialities of attention mechanisms presented in the literature. It proposes future research lines in medical applications that may benefit from these frameworks.
• Score: 2.8101673772585736
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The increasing popularity of attention mechanisms in deep learning algorithms for computer vision and natural language processing made these models attractive to other research domains. In healthcare, there is a strong need for tools that may improve the routines of the clinicians and the patients. Naturally, the use of attention-based algorithms for medical applications occurred smoothly. However, being healthcare a domain that depends on high-stake decisions, the scientific community must ponder if these high-performing algorithms fit the needs of medical applications. With this motto, this paper extensively reviews the use of attention mechanisms in machine learning (including Transformers) for several medical applications. This work distinguishes itself from its predecessors by proposing a critical analysis of the claims and potentialities of attention mechanisms presented in the literature through an experimental case study on medical image classification with three different use cases. These experiments focus on the integrating process of attention mechanisms into established deep learning architectures, the analysis of their predictive power, and a visual assessment of their saliency maps generated by post-hoc explanation methods. This paper concludes with a critical analysis of the claims and potentialities presented in the literature about attention mechanisms and proposes future research lines in medical applications that may benefit from these frameworks.

Related papers

• Machine Learning Innovations in CPR: A Comprehensive Survey on Enhanced Resuscitation Techniques [52.71395121577439]
  This survey paper explores the transformative role of Machine Learning (ML) and Artificial Intelligence (AI) in Cardiopulmonary Resuscitation (CPR) It highlights the impact of predictive modeling, AI-enhanced devices, and real-time data analysis in improving resuscitation outcomes. The paper provides a comprehensive overview, classification, and critical analysis of current applications, challenges, and future directions in this emerging field.
  arXiv  Detail & Related papers  (2024-11-03T18:01:50Z)
• Exploration of Attention Mechanism-Enhanced Deep Learning Models in the Mining of Medical Textual Data [3.22071437711162]
  The research explores the utilization of a deep learning model employing an attention mechanism in medical text mining. It aims to enhance the model's capability to identify essential medical information by incorporating deep learning and attention mechanisms.
  arXiv  Detail & Related papers  (2024-05-23T00:20:14Z)
• Brain-Inspired Machine Intelligence: A Survey of Neurobiologically-Plausible Credit Assignment [65.268245109828]
  We examine algorithms for conducting credit assignment in artificial neural networks that are inspired or motivated by neurobiology. We organize the ever-growing set of brain-inspired learning schemes into six general families and consider these in the context of backpropagation of errors. The results of this review are meant to encourage future developments in neuro-mimetic systems and their constituent learning processes.
  arXiv  Detail & Related papers  (2023-12-01T05:20:57Z)
• Tracking and Mapping in Medical Computer Vision: A Review [23.28261994515735]
  As computer vision algorithms increase in capability, their applications in clinical systems will become more pervasive. These applications include: diagnostics, such as colonoscopy and bronchoscopy; guiding biopsies, minimally invasive interventions, and surgery. Many of these applications depend on the specific visual nature of medical scenes and require designing algorithms to perform in this environment.
  arXiv  Detail & Related papers  (2023-10-17T03:36:28Z)
• Attention Mechanisms in Medical Image Segmentation: A Survey [41.093501592520326]
  This paper systematically reviews the basic principles of attention mechanisms and their applications in medical image segmentation. We surveyed over 300 articles related to medical image segmentation, and divided them into two groups based on their attention mechanisms, non-Transformer attention and Transformer attention.
  arXiv  Detail & Related papers  (2023-05-29T08:00:54Z)
• Studying the Effects of Self-Attention for Medical Image Analysis [42.12044020360494]
  We compare various state-of-the-art self-attention mechanisms across multiple medical image analysis tasks. We aim to provide a deeper understanding of the effects of self-attention in medical computer vision tasks.
  arXiv  Detail & Related papers  (2021-09-02T07:07:16Z)
• Domain Shift in Computer Vision models for MRI data analysis: An Overview [64.69150970967524]
  Machine learning and computer vision methods are showing good performance in medical imagery analysis. Yet only a few applications are now in clinical use. Poor transferability of themodels to data from different sources or acquisition domains is one of the reasons for that.
  arXiv  Detail & Related papers  (2020-10-14T16:34:21Z)
• Extracting a Knowledge Base of Mechanisms from COVID-19 Papers [50.17242035034729]
  We pursue the construction of a knowledge base (KB) of mechanisms. We develop a broad, unified schema that strikes a balance between relevance and breadth. Experiments demonstrate the utility of our KB in supporting interdisciplinary scientific search over COVID-19 literature.
  arXiv  Detail & Related papers  (2020-10-08T07:54:14Z)
• Machine Learning in Nano-Scale Biomedical Engineering [77.75587007080894]
  We review the existing research regarding the use of machine learning in nano-scale biomedical engineering. The main challenges that can be formulated as ML problems are classified into the three main categories. For each of the presented methodologies, special emphasis is given to its principles, applications, and limitations.
  arXiv  Detail & Related papers  (2020-08-05T15:45:54Z)
• Machine learning-based clinical prediction modeling -- A practical guide for clinicians [0.0]
  The number of manuscripts related to machine learning or artificial intelligence has exponentially increased over the past years. In the first section, we provide explanations on the general principles of machine learning. In further sections, we review the importance of resampling, overfitting and model generalizability and strategies for model evaluation.
  arXiv  Detail & Related papers  (2020-06-23T20:11:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.