HeartBERT: A Self-Supervised ECG Embedding Model for Efficient and Effective Medical Signal Analysis

• URL: http://arxiv.org/abs/2411.11896v1
• Date: Fri, 08 Nov 2024 14:25:00 GMT
• Title: HeartBERT: A Self-Supervised ECG Embedding Model for Efficient and Effective Medical Signal Analysis
• Authors: Saedeh Tahery, Fatemeh Hamid Akhlaghi, Termeh Amirsoleimani, Saeed Farzi,
• Abstract summary: HeartBert is inspired by Bidirectional Representations from Transformers (BERT) in natural language processing and enhanced with a self-supervised learning approach. To demonstrate the versatility, generalizability, and efficiency of the proposed model, two key downstream tasks have been selected: sleep stage detection and heartbeat classification. A series of practical experiments have been conducted to demonstrate the superiority and advancements of HeartBERT.
• Score: 1.124958340749622
• License:
• Abstract: The HeartBert model is introduced with three primary objectives: reducing the need for labeled data, minimizing computational resources, and simultaneously improving performance in machine learning systems that analyze Electrocardiogram (ECG) signals. Inspired by Bidirectional Encoder Representations from Transformers (BERT) in natural language processing and enhanced with a self-supervised learning approach, the HeartBert model-built on the RoBERTa architecture-generates sophisticated embeddings tailored for ECG-based projects in the medical domain. To demonstrate the versatility, generalizability, and efficiency of the proposed model, two key downstream tasks have been selected: sleep stage detection and heartbeat classification. HeartBERT-based systems, utilizing bidirectional LSTM heads, are designed to address complex challenges. A series of practical experiments have been conducted to demonstrate the superiority and advancements of HeartBERT, particularly in terms of its ability to perform well with smaller training datasets, reduced learning parameters, and effective performance compared to rival models. The code and data are publicly available at https://github.com/ecgResearch/HeartBert.

Related papers

• ConvexECG: Lightweight and Explainable Neural Networks for Personalized, Continuous Cardiac Monitoring [43.23305904110984]
  ConvexECG is an explainable and resource-efficient method for reconstructing six-lead electrocardiograms from single-lead data. We demonstrate that ConvexECG achieves accuracy comparable to larger neural networks while significantly reducing computational overhead.
  arXiv  Detail & Related papers  (2024-09-19T06:14:30Z)
• ECG Arrhythmia Detection Using Disease-specific Attention-based Deep Learning Model [0.0]
  We propose a disease-specific attention-based deep learning model (DANet) for arrhythmia detection from short ECG recordings. The novel idea is to introduce a soft-coding or hard-coding waveform enhanced module into existing deep neural networks. For the soft-coding DANet, we also develop a learning framework combining self-supervised pre-training with two-stage supervised training.
  arXiv  Detail & Related papers  (2024-07-25T13:27:10Z)
• Automatic Cardiac Pathology Recognition in Echocardiography Images Using Higher Order Dynamic Mode Decomposition and a Vision Transformer for Small Datasets [2.0286377328378737]
  Heart diseases are the main international cause of human defunction. According to the WHO, nearly 18 million people decease each year because of heart diseases. In this work, an automatic cardiac pathology recognition system based on a novel deep learning framework is proposed.
  arXiv  Detail & Related papers  (2024-04-30T14:16:45Z)
• Hierarchical Deep Learning with Generative Adversarial Network for Automatic Cardiac Diagnosis from ECG Signals [2.5008947886814186]
  We propose a two-level hierarchical deep learning framework with Generative Adversarial Network (GAN) for automatic diagnosis of ECG signals. The first-level model is composed of a Memory-Augmented Deep auto-Encoder with GAN, which aims to differentiate abnormal signals from normal ECGs for anomaly detection. The second-level learning aims at robust multi-class classification for different arrhythmias identification.
  arXiv  Detail & Related papers  (2022-10-19T12:29:05Z)
• Generalizing electrocardiogram delineation: training convolutional neural networks with synthetic data augmentation [63.51064808536065]
  Existing databases for ECG delineation are small, being insufficient in size and in the array of pathological conditions they represent. This article delves has two main contributions. First, a pseudo-synthetic data generation algorithm was developed, based in probabilistically composing ECG traces given "pools" of fundamental segments, as cropped from the original databases, and a set of rules for their arrangement into coherent synthetic traces. Second, two novel segmentation-based loss functions have been developed, which attempt at enforcing the prediction of an exact number of independent structures and at producing closer segmentation boundaries by focusing on a reduced number of samples.
  arXiv  Detail & Related papers  (2021-11-25T10:11:41Z)
• A multi-stage machine learning model on diagnosis of esophageal manometry [50.591267188664666]
  The framework includes deep-learning models at the swallow-level stage and feature-based machine learning models at the study-level stage. This is the first artificial-intelligence-style model to automatically predict CC diagnosis of HRM study from raw multi-swallow data.
  arXiv  Detail & Related papers  (2021-06-25T20:09:23Z)
• Heart-Darts: Classification of Heartbeats Using Differentiable Architecture Search [22.225051965963114]
  Arrhythmia is a cardiovascular disease that manifests irregular heartbeats. In arrhythmia detection, the electrocardiogram (ECG) signal is an important diagnostic technique. With the application of convolutional neural networks (CNNs), the evaluation process has been accelerated and the performance is improved.
  arXiv  Detail & Related papers  (2021-05-03T08:57:48Z)
• On Energy-Based Models with Overparametrized Shallow Neural Networks [44.74000986284978]
  Energy-based models (EBMs) are a powerful framework for generative modeling. In this work we focus on shallow neural networks. We show that models trained in the so-called "active" regime provide a statistical advantage over their associated "lazy" or kernel regime.
  arXiv  Detail & Related papers  (2021-04-15T15:34:58Z)
• MetaDistiller: Network Self-Boosting via Meta-Learned Top-Down Distillation [153.56211546576978]
  In this work, we propose that better soft targets with higher compatibil-ity can be generated by using a label generator. We can employ the meta-learning technique to optimize this label generator. The experiments are conducted on two standard classificationbenchmarks, namely CIFAR-100 and ILSVRC2012.
  arXiv  Detail & Related papers  (2020-08-27T13:04:27Z)
• ECG-DelNet: Delineation of Ambulatory Electrocardiograms with Mixed Quality Labeling Using Neural Networks [69.25956542388653]
  Deep learning (DL) algorithms are gaining weight in academic and industrial settings. We demonstrate DL can be successfully applied to low interpretative tasks by embedding ECG detection and delineation onto a segmentation framework. The model was trained using PhysioNet's QT database, comprised of 105 ambulatory ECG recordings.
  arXiv  Detail & Related papers  (2020-05-11T16:29:12Z)
• Multi-Lead ECG Classification via an Information-Based Attention Convolutional Neural Network [1.1720399305661802]
  One-dimensional convolutional neural networks (CNN) have proven to be effective in pervasive classification tasks. We implement the Residual connection and design a structure which can learn the weights from the information contained in different channels in the input feature map. An indicator named mean square deviation is introduced to monitor the performance of a particular model segment in the classification task.
  arXiv  Detail & Related papers  (2020-03-25T02:28:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.