Interpretable Pre-Trained Transformers for Heart Time-Series Data

• URL: http://arxiv.org/abs/2407.20775v2
• Date: Tue, 13 Aug 2024 10:18:45 GMT
• Title: Interpretable Pre-Trained Transformers for Heart Time-Series Data
• Authors: Harry J. Davies, James Monsen, Danilo P. Mandic,
• Abstract summary: We create two pre-trained general purpose cardiac models, PPG-PT and ECG-PT. We highlight that individual attention heads respond to specific physiologically relevent features, such as the dicrotic notch in PPG and the P-wave in ECG. These pre-trained models are straightforward to fine-tune for tasks such as classification of atrial fibrillation (AF), and beat detection in photoplethysmography.
• Score: 15.377534937558744
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Decoder-only transformers are the backbone of the popular generative pre-trained transformer (GPT) series of large language models. In this work, we employ this framework to the analysis of clinical heart time-series data, to create two pre-trained general purpose cardiac models, termed PPG-PT and ECG-PT. We place a special emphasis on making both such pre-trained models fully interpretable. This is achieved firstly through aggregate attention maps which show that, in order to make predictions, the model focuses on similar points in previous cardiac cycles and gradually broadens its attention in deeper layers. Next, we show that tokens with the same value, which occur at different distinct points in the electrocardiography (ECG) and photoplethysmography (PPG) cycle, form separate clusters in high dimensional space. The clusters form according to phase, as the tokens propagate through the transformer blocks. Finally, we highlight that individual attention heads respond to specific physiologically relevent features, such as the dicrotic notch in PPG and the P-wave in ECG. It is also demonstrated that these pre-trained models are straightforward to fine-tune for tasks such as classification of atrial fibrillation (AF), and beat detection in photoplethysmography. For the example of AF, the fine-tuning took 11 minutes of computer time, and achieved the respective leave-one-subject-out AUCs of 0.99 and 0.93 for ECG and PPG within the MIMIC Perform AF dataset. In addition, the fine-tuned beat detector achieved a state-of-the-art F1 score of 98%, as well as uniquely providing a beat confidence level which acts as a signal quality estimator. Importantly, the fine-tuned models for AF screening are also fully explainable, with attention shifting to regions in the context that are strongly indicative of atrial fibrillation.

Related papers

• Self-supervised inter-intra period-aware ECG representation learning for detecting atrial fibrillation [41.82319894067087]
  We propose an inter-intra period-aware ECG representation learning approach. Considering ECGs of atrial fibrillation patients exhibit the irregularity in RR intervals and the absence of P-waves, we develop specific pre-training tasks for interperiod and intraperiod representations. Our approach demonstrates remarkable AUC performances on the BTCH dataset, textiti.e., 0.953/0.996 for paroxysmal/persistent atrial fibrillation detection.
  arXiv  Detail & Related papers  (2024-10-08T10:03:52Z)
• Improving Diffusion Models for ECG Imputation with an Augmented Template Prior [43.6099225257178]
  noisy and poor-quality recordings are a major issue for signals collected using mobile health systems. Recent studies have explored the imputation of missing values in ECG with probabilistic time-series models. We present a template-guided denoising diffusion probabilistic model (DDPM), PulseDiff, which is conditioned on an informative prior for a range of health conditions.
  arXiv  Detail & Related papers  (2023-10-24T11:34:15Z)
• PulseNet: Deep Learning ECG-signal classification using random augmentation policy and continous wavelet transform for canines [46.09869227806991]
  evaluating canine electrocardiograms (ECG) require skilled veterinarians. Current availability of veterinary cardiologists for ECG interpretation and diagnostic support is limited. We implement a deep convolutional neural network (CNN) approach for classifying canine electrocardiogram sequences as either normal or abnormal.
  arXiv  Detail & Related papers  (2023-05-17T09:06:39Z)
• Deep learning based ECG segmentation for delineation of diverse arrhythmias [0.0]
  We introduce a U-Net-like segmentation model for ECG delineation with a particular focus on diverse arrhythmias. This is followed by a post-processing algorithm which removes noise and automatically determines the boundaries of P, QRS, and T waves. Our model has been trained on a diverse dataset and evaluated against the LUDB and QTDB datasets to show strong performance, with F1-scores exceeding 99% for QRS and T waves, and over 97% for P waves in the LUDB dataset.
  arXiv  Detail & Related papers  (2023-04-13T03:20:45Z)
• Performer: A Novel PPG to ECG Reconstruction Transformer For a Digital Biomarker of Cardiovascular Disease Detection [0.0]
  Cardiovascular diseases (CVDs) have become the top one cause of death; three-quarters of these deaths occur in lower-income communities. Electrocardiography (ECG) is infeasible for continuous cardiac monitoring due to its requirement for user participation. Photoplethysmography is easy to collect, but the limited accuracy constrains its clinical usage.
  arXiv  Detail & Related papers  (2022-04-25T17:10:13Z)
• Global ECG Classification by Self-Operational Neural Networks with Feature Injection [25.15075119957447]
  We propose a novel approach for inter-patient ECG classification using a compact 1D Self-Organized Operational Neural Networks (Self-ONNs) We used 1D Self-ONN layers to automatically learn morphological representations from ECG data, enabling us to capture the shape of the ECG waveform around the R peaks. Using the MIT-BIH arrhythmia benchmark database, the proposed method achieves the highest classification performance ever achieved.
  arXiv  Detail & Related papers  (2022-04-07T22:49:18Z)
• Generalizing electrocardiogram delineation: training convolutional neural networks with synthetic data augmentation [63.51064808536065]
  Existing databases for ECG delineation are small, being insufficient in size and in the array of pathological conditions they represent. This article delves has two main contributions. First, a pseudo-synthetic data generation algorithm was developed, based in probabilistically composing ECG traces given "pools" of fundamental segments, as cropped from the original databases, and a set of rules for their arrangement into coherent synthetic traces. Second, two novel segmentation-based loss functions have been developed, which attempt at enforcing the prediction of an exact number of independent structures and at producing closer segmentation boundaries by focusing on a reduced number of samples.
  arXiv  Detail & Related papers  (2021-11-25T10:11:41Z)
• Anomaly Detection in Time Series with Triadic Motif Fields and Application in Atrial Fibrillation ECG Classification [3.026059658770843]
  In the time-series analysis, the time series motifs and the order patterns in time series can reveal general temporal patterns and dynamic features. TMF is a simple and effective time-series image encoding method based on triadic time series motifs. The dynamic features can be extracted from the TMF images with the transfer learning pre-trained convolutional neural network (CNN) models. The patterns revealed by symmetrized Gradient-weighted Class Mapping have a clear clinical interpretation.
  arXiv  Detail & Related papers  (2020-12-09T09:39:48Z)
• Multilabel 12-Lead Electrocardiogram Classification Using Gradient Boosting Tree Ensemble [64.29529357862955]
  We build an algorithm using gradient boosted tree ensembles fitted on morphology and signal processing features to classify ECG diagnosis. For each lead, we derive features from heart rate variability, PQRST template shape, and the full signal waveform. We join the features of all 12 leads to fit an ensemble of gradient boosting decision trees to predict probabilities of ECG instances belonging to each class.
  arXiv  Detail & Related papers  (2020-10-21T18:11:36Z)
• Classification of Arrhythmia by Using Deep Learning with 2-D ECG Spectral Image Representation [3.3426603061273994]
  We propose a two-dimensional (2-D) convolutional neural network (CNN) model for the classification of ECG signals into eight classes. We achieved a state-of-the-art average classification accuracy of 99.11%, which is better than those of recently reported results in classifying similar types of arrhythmias.
  arXiv  Detail & Related papers  (2020-05-14T12:11:41Z)
• ECG-DelNet: Delineation of Ambulatory Electrocardiograms with Mixed Quality Labeling Using Neural Networks [69.25956542388653]
  Deep learning (DL) algorithms are gaining weight in academic and industrial settings. We demonstrate DL can be successfully applied to low interpretative tasks by embedding ECG detection and delineation onto a segmentation framework. The model was trained using PhysioNet's QT database, comprised of 105 ambulatory ECG recordings.
  arXiv  Detail & Related papers  (2020-05-11T16:29:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.