Echo-SyncNet: Self-supervised Cardiac View Synchronization in Echocardiography

• URL: http://arxiv.org/abs/2102.02287v1
• Date: Wed, 3 Feb 2021 20:48:16 GMT
• Title: Echo-SyncNet: Self-supervised Cardiac View Synchronization in Echocardiography
• Authors: Fatemeh Taheri Dezaki, Christina Luong, Tom Ginsberg, Robert Rohling, Ken Gin, Purang Abolmaesumi, Teresa Tsang
• Abstract summary: We propose Echo-Sync-Net, a self-supervised learning framework to synchronize various cross-of-care 2D echo series without any external input. We show promising results for synchronizing Apical 2 chamber and Apical 4 chamber cardiac views. We also show the usefulness of the learned representations in a one-shot learning scenario of cardiac detection.
• Score: 11.407910072022018
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: In echocardiography (echo), an electrocardiogram (ECG) is conventionally used to temporally align different cardiac views for assessing critical measurements. However, in emergencies or point-of-care situations, acquiring an ECG is often not an option, hence motivating the need for alternative temporal synchronization methods. Here, we propose Echo-SyncNet, a self-supervised learning framework to synchronize various cross-sectional 2D echo series without any external input. The proposed framework takes advantage of both intra-view and inter-view self supervisions. The former relies on spatiotemporal patterns found between the frames of a single echo cine and the latter on the interdependencies between multiple cines. The combined supervisions are used to learn a feature-rich embedding space where multiple echo cines can be temporally synchronized. We evaluate the framework with multiple experiments: 1) Using data from 998 patients, Echo-SyncNet shows promising results for synchronizing Apical 2 chamber and Apical 4 chamber cardiac views; 2) Using data from 3070 patients, our experiments reveal that the learned representations of Echo-SyncNet outperform a supervised deep learning method that is optimized for automatic detection of fine-grained cardiac phase; 3) We show the usefulness of the learned representations in a one-shot learning scenario of cardiac keyframe detection. Without any fine-tuning, keyframes in 1188 validation patient studies are identified by synchronizing them with only one labeled reference study. We do not make any prior assumption about what specific cardiac views are used for training and show that Echo-SyncNet can accurately generalize to views not present in its training set. Project repository: github.com/fatemehtd/Echo-SyncNet.

Related papers

• HeartBeat: Towards Controllable Echocardiography Video Synthesis with Multimodal Conditions-Guided Diffusion Models [14.280181445804226]
  We propose a novel framework named HeartBeat towards controllable and high-fidelity ECHO video synthesis. HeartBeat serves as a unified framework that enables perceiving multimodal conditions simultaneously to guide controllable generation. In this way, users can synthesize ECHO videos that conform to their mental imagery by combining multimodal control signals.
  arXiv  Detail & Related papers  (2024-06-20T08:24:28Z)
• Synchformer: Efficient Synchronization from Sparse Cues [100.89656994681934]
  Our contributions include a novel audio-visual synchronization model, and training that decouples extraction from synchronization modelling. This approach achieves state-of-the-art performance in both dense and sparse settings. We also extend synchronization model training to AudioSet a million-scale 'in-the-wild' dataset, investigate evidence attribution techniques for interpretability, and explore a new capability for synchronization models: audio-visual synchronizability.
  arXiv  Detail & Related papers  (2024-01-29T18:59:55Z)
• GestSync: Determining who is speaking without a talking head [67.75387744442727]
  We introduce Gesture-Sync: determining if a person's gestures are correlated with their speech or not. In comparison to Lip-Sync, Gesture-Sync is far more challenging as there is a far looser relationship between the voice and body movement. We show that the model can be trained using self-supervised learning alone, and evaluate its performance on the LRS3 dataset.
  arXiv  Detail & Related papers  (2023-10-08T22:48:30Z)
• SimLVSeg: Simplifying Left Ventricular Segmentation in 2D+Time Echocardiograms with Self- and Weakly-Supervised Learning [0.8672882547905405]
  We develop SimLVSeg, a video-based network for consistent left ventricular (LV) segmentation from sparsely annotated echocardiogram videos. SimLVSeg consists of self-supervised pre-training with temporal masking, followed by weakly supervised learning tailored for LV segmentation from sparse annotations. We demonstrate how SimLVSeg outperforms the state-of-the-art solutions by achieving a 93.32% dice score on the largest 2D+time echocardiography dataset.
  arXiv  Detail & Related papers  (2023-09-30T18:13:41Z)
• Extraction of volumetric indices from echocardiography: which deep learning solution for clinical use? [6.144041824426555]
  We show that the proposed 3D nnU-Net outperforms alternative 2D and recurrent segmentation methods. Overall, the experimental results suggest that with sufficient training data, 3D nnU-Net could become the first automated tool to meet the standards of an everyday clinical device.
  arXiv  Detail & Related papers  (2023-05-03T09:38:52Z)
• Sparse in Space and Time: Audio-visual Synchronisation with Trainable Selectors [103.21152156339484]
  The objective of this paper is audio-visual synchronisation of general videos 'in the wild' We make four contributions: (i) in order to handle longer temporal sequences required for sparse synchronisation signals, we design a multi-modal transformer model that employs'selectors' We identify artefacts that can arise from the compression codecs used for audio and video and can be used by audio-visual models in training to artificially solve the synchronisation task.
  arXiv  Detail & Related papers  (2022-10-13T14:25:37Z)
• Echocardiography Segmentation with Enforced Temporal Consistency [10.652677452009627]
  We propose a framework to learn the 2D+time long-axis cardiac shape. The identification and correction of cardiac inconsistencies relies on a constrained autoencoder trained to learn a physiologically interpretable embedding of cardiac shapes.
  arXiv  Detail & Related papers  (2021-12-03T16:09:32Z)
• Multi-view Temporal Alignment for Non-parallel Articulatory-to-Acoustic Speech Synthesis [59.623780036359655]
  Articulatory-to-acoustic (A2A) synthesis refers to the generation of audible speech from captured movement of the speech articulators. This technique has numerous applications, such as restoring oral communication to people who cannot longer speak due to illness or injury. We propose a solution to this problem based on the theory of multi-view learning.
  arXiv  Detail & Related papers  (2020-12-30T15:09:02Z)
• Noise-Resilient Automatic Interpretation of Holter ECG Recordings [67.59562181136491]
  We present a three-stage process for analysing Holter recordings with robustness to noisy signal. First stage is a segmentation neural network (NN) with gradientdecoder architecture which detects positions of heartbeats. Second stage is a classification NN which will classify heartbeats as wide or narrow. Third stage is a boosting decision trees (GBDT) on top of NN features that incorporates patient-wise features.
  arXiv  Detail & Related papers  (2020-11-17T16:15:49Z)
• End-to-End Deep Learning Model for Cardiac Cycle Synchronization from Multi-View Angiographic Sequences [3.4377441151536376]
  Temporal matching of the different views, which may not be acquired simultaneously, is a prerequisite for an accurate stereo-matching of the coronary segments. We show how a neural network can be trained from angiographic sequences to synchronize different views during the cardiac cycle.
  arXiv  Detail & Related papers  (2020-09-04T18:11:50Z)
• AutoHR: A Strong End-to-end Baseline for Remote Heart Rate Measurement with Neural Searching [76.4844593082362]
  We investigate the reason why existing end-to-end networks perform poorly in challenging conditions and establish a strong baseline for remote HR measurement with architecture search (NAS) Comprehensive experiments are performed on three benchmark datasets on both intra-temporal and cross-dataset testing.
  arXiv  Detail & Related papers  (2020-04-26T05:43:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.