Hierarchical Transformer for Electrocardiogram Diagnosis

• URL: http://arxiv.org/abs/2411.00755v1
• Date: Fri, 01 Nov 2024 17:28:03 GMT
• Title: Hierarchical Transformer for Electrocardiogram Diagnosis
• Authors: Xiaoya Tang, Jake Berquist, Benjamin A. Steinberg, Tolga Tasdizen,
• Abstract summary: Transformers, originally prominent in NLP and computer vision, are now being adapted for ECG signal analysis. This paper introduces a novel hierarchical transformer architecture that segments the model into multiple stages. A classification token aggregates information across feature scales, facilitating interactions between different stages of the transformer.
• Score: 1.4124476944967472
• License:
• Abstract: Transformers, originally prominent in NLP and computer vision, are now being adapted for ECG signal analysis. This paper introduces a novel hierarchical transformer architecture that segments the model into multiple stages by assessing the spatial size of the embeddings, thus eliminating the need for additional downsampling strategies or complex attention designs. A classification token aggregates information across feature scales, facilitating interactions between different stages of the transformer. By utilizing depth-wise convolutions in a six-layer convolutional encoder, our approach preserves the relationships between different ECG leads. Moreover, an attention gate mechanism learns associations among the leads prior to classification. This model adapts flexibly to various embedding networks and input sizes while enhancing the interpretability of transformers in ECG signal analysis.

Related papers

• ECG Signal Denoising Using Multi-scale Patch Embedding and Transformers [6.882042556551613]
  We propose a deep learning method that combines a one-dimensional convolutional layer with transformer architecture for denoising ECG signals. The embedding then is used as the input of a transformer network and enhances the capability of the transformer for denoising the ECG signal.
  arXiv  Detail & Related papers  (2024-07-12T03:13:52Z)
• Automatic Graph Topology-Aware Transformer [50.2807041149784]
  We build a comprehensive graph Transformer search space with the micro-level and macro-level designs. EGTAS evolves graph Transformer topologies at the macro level and graph-aware strategies at the micro level. We demonstrate the efficacy of EGTAS across a range of graph-level and node-level tasks.
  arXiv  Detail & Related papers  (2024-05-30T07:44:31Z)
• Rethinking Attention Gated with Hybrid Dual Pyramid Transformer-CNN for Generalized Segmentation in Medical Imaging [17.07490339960335]
  We introduce a novel hybrid CNN-Transformer segmentation architecture (PAG-TransYnet) designed for efficiently building a strong CNN-Transformer encoder. Our approach exploits attention gates within a Dual Pyramid hybrid encoder.
  arXiv  Detail & Related papers  (2024-04-28T14:37:10Z)
• Affine-Consistent Transformer for Multi-Class Cell Nuclei Detection [76.11864242047074]
  We propose a novel Affine-Consistent Transformer (AC-Former), which directly yields a sequence of nucleus positions. We introduce an Adaptive Affine Transformer (AAT) module, which can automatically learn the key spatial transformations to warp original images for local network training. Experimental results demonstrate that the proposed method significantly outperforms existing state-of-the-art algorithms on various benchmarks.
  arXiv  Detail & Related papers  (2023-10-22T02:27:02Z)
• Multi-scale Transformer-based Network for Emotion Recognition from Multi Physiological Signals [11.479653866646762]
  This paper presents an efficient Multi-scale Transformer-based approach for the task of Emotion recognition from Physiological data. Our approach involves applying a Multi-modal technique combined with scaling data to establish the relationship between internal body signals and human emotions. Our model achieves decent results on the CASE dataset of the EPiC competition, with an RMSE score of 1.45.
  arXiv  Detail & Related papers  (2023-05-01T11:10:48Z)
• A Dual-scale Lead-seperated Transformer With Lead-orthogonal Attention And Meta-information For Ecg Classification [26.07181634056045]
  This work proposes a dual-scale lead-separated transformer with lead-orthogonal attention and meta-information (DLTM-ECG) ECG segments are interpreted as independent patches, and together with the reduced dimension signal, they form a dual-scale representation. Our work has the potential for similar multichannel bioelectrical signal processing and physiological multimodal tasks.
  arXiv  Detail & Related papers  (2022-11-23T08:45:34Z)
• SIM-Trans: Structure Information Modeling Transformer for Fine-grained Visual Categorization [59.732036564862796]
  We propose the Structure Information Modeling Transformer (SIM-Trans) to incorporate object structure information into transformer for enhancing discriminative representation learning. The proposed two modules are light-weighted and can be plugged into any transformer network and trained end-to-end easily. Experiments and analyses demonstrate that the proposed SIM-Trans achieves state-of-the-art performance on fine-grained visual categorization benchmarks.
  arXiv  Detail & Related papers  (2022-08-31T03:00:07Z)
• Cost Aggregation with 4D Convolutional Swin Transformer for Few-Shot Segmentation [58.4650849317274]
  Volumetric Aggregation with Transformers (VAT) is a cost aggregation network for few-shot segmentation. VAT attains state-of-the-art performance for semantic correspondence as well, where cost aggregation also plays a central role.
  arXiv  Detail & Related papers  (2022-07-22T04:10:30Z)
• Exploring Structure-aware Transformer over Interaction Proposals for Human-Object Interaction Detection [119.93025368028083]
  We design a novel Transformer-style Human-Object Interaction (HOI) detector, i.e., Structure-aware Transformer over Interaction Proposals (STIP) STIP decomposes the process of HOI set prediction into two subsequent phases, i.e., an interaction proposal generation is first performed, and then followed by transforming the non-parametric interaction proposals into HOI predictions via a structure-aware Transformer. The structure-aware Transformer upgrades vanilla Transformer by encoding additionally the holistically semantic structure among interaction proposals as well as the locally spatial structure of human/object within each interaction proposal, so as to strengthen HOI
  arXiv  Detail & Related papers  (2022-06-13T16:21:08Z)
• Combining Transformer Generators with Convolutional Discriminators [9.83490307808789]
  Recently proposed TransGAN is the first GAN using only transformer-based architectures. TransGAN requires data augmentation, an auxiliary super-resolution task during training, and a masking prior to guide the self-attention mechanism. We evaluate our approach by conducting a benchmark of well-known CNN discriminators, ablate the size of the transformer-based generator, and show that combining both architectural elements into a hybrid model leads to better results.
  arXiv  Detail & Related papers  (2021-05-21T07:56:59Z)
• Multi-Pass Transformer for Machine Translation [51.867982400693194]
  We consider a multi-pass transformer (MPT) architecture in which earlier layers are allowed to process information in light of the output of later layers. MPT can surpass the performance of Large Transformer on the challenging machine translation En-De and En-Fr datasets. In the hard connection case, the optimal connection pattern found for En-De also leads to improved performance for En-Fr.
  arXiv  Detail & Related papers  (2020-09-23T21:22:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.