Fusing Echocardiography Images and Medical Records for Continuous
Patient Stratification
- URL: http://arxiv.org/abs/2401.07796v1
- Date: Mon, 15 Jan 2024 16:04:46 GMT
- Title: Fusing Echocardiography Images and Medical Records for Continuous
Patient Stratification
- Authors: Nathan Painchaud, Pierre-Yves Courand, Pierre-Marc Jodoin, Nicolas
Duchateau, Olivier Bernard
- Abstract summary: We propose a method that considers all descriptors extracted from medical records and echocardiograms to learn the representation of a difficult-to-characterize cardiovascular pathology, namely hypertension.
Our method first projects each variable into its own representation space using modality-specific approaches.
These standardized representations of multimodal data are then fed to a transformer encoder, which learns to merge them into a comprehensive representation of the patient through a pretext task of predicting a clinical rating.
- Score: 6.328889967237029
- License: http://creativecommons.org/licenses/by/4.0/
- Abstract: Deep learning now enables automatic and robust extraction of cardiac function
descriptors from echocardiographic sequences, such as ejection fraction or
strain. These descriptors provide fine-grained information that physicians
consider, in conjunction with more global variables from the clinical record,
to assess patients' condition. Drawing on novel transformer models applied to
tabular data (e.g., variables from electronic health records), we propose a
method that considers all descriptors extracted from medical records and
echocardiograms to learn the representation of a difficult-to-characterize
cardiovascular pathology, namely hypertension. Our method first projects each
variable into its own representation space using modality-specific approaches.
These standardized representations of multimodal data are then fed to a
transformer encoder, which learns to merge them into a comprehensive
representation of the patient through a pretext task of predicting a clinical
rating. This pretext task is formulated as an ordinal classification to enforce
a pathological continuum in the representation space. We observe the major
trends along this continuum for a cohort of 239 hypertensive patients to
describe, with unprecedented gradation, the effect of hypertension on a number
of cardiac function descriptors. Our analysis shows that i) pretrained weights
from a foundation model allow to reach good performance (83% accuracy) even
with limited data (less than 200 training samples), ii) trends across the
population are reproducible between trainings, and iii) for descriptors whose
interactions with hypertension are well documented, patterns are consistent
with prior physiological knowledge.
Related papers
- SHAPE: A Sample-adaptive Hierarchical Prediction Network for Medication
Recommendation [22.899946140205962]
We propose a novel Sample-adaptive Hierarchical medicAtion Prediction nEtwork, termed SHAPE, to tackle the challenges in the medication recommendation task.
Specifically, we design a compact intra-visit set encoder to encode the relationship in the medical event for obtaining visit-level representation.
To endow the model with the capability of modeling the variable visit length, we introduce a soft curriculum learning method to assign the difficulty of each sample automatically by the visit length.
arXiv Detail & Related papers (2023-09-09T08:28:04Z) - Hypergraph Convolutional Networks for Fine-grained ICU Patient
Similarity Analysis and Risk Prediction [15.06049250330114]
The Intensive Care Unit (ICU) is one of the most important parts of a hospital, which admits critically ill patients and provides continuous monitoring and treatment.
Various patient outcome prediction methods have been attempted to assist healthcare professionals in clinical decision-making.
arXiv Detail & Related papers (2023-08-24T05:26:56Z) - A Conditional Flow Variational Autoencoder for Controllable Synthesis of
Virtual Populations of Anatomy [76.20367415712867]
We propose a conditional variational autoencoder (cVAE) with normalising flows to boost the flexibility and complexity of the approximate posterior learnt.
We demonstrate the performance of our conditional flow VAE using a data set of cardiac left ventricles acquired from 2360 patients.
arXiv Detail & Related papers (2023-06-26T13:23:52Z) - Mesh Neural Networks for SE(3)-Equivariant Hemodynamics Estimation on the Artery Wall [13.113110989699571]
We consider the estimation of vector-valued quantities on the wall of three-dimensional geometric artery models.
We employ group equivariant graph convolution in an end-to-end SE(3)-equivariant neural network that operates directly on triangular surface meshes.
We show that our method is powerful enough to accurately predict transient, vector-valued WSS over the cardiac cycle while conditioned on a range of different inflow boundary conditions.
arXiv Detail & Related papers (2022-12-09T18:16:06Z) - Towards dynamic multi-modal phenotyping using chest radiographs and
physiological data [3.11179491890629]
We propose a dynamic training approach to learn modality-specific data representations and to integrate auxiliary features.
Preliminary experiments results for a patient phenotyping task using physiological data in MIMIC-IV & chest radiographs in the MIMIC- CXR dataset.
This illustrates the benefit of leveraging the chest imaging modality in the phenotyping task and highlights the potential of multi-modal learning in medical applications.
arXiv Detail & Related papers (2021-11-04T09:41:00Z) - A Deep Learning Approach to Predicting Collateral Flow in Stroke
Patients Using Radiomic Features from Perfusion Images [58.17507437526425]
Collateral circulation results from specialized anastomotic channels which provide oxygenated blood to regions with compromised blood flow.
The actual grading is mostly done through manual inspection of the acquired images.
We present a deep learning approach to predicting collateral flow grading in stroke patients based on radiomic features extracted from MR perfusion data.
arXiv Detail & Related papers (2021-10-24T18:58:40Z) - Noise-Resilient Automatic Interpretation of Holter ECG Recordings [67.59562181136491]
We present a three-stage process for analysing Holter recordings with robustness to noisy signal.
First stage is a segmentation neural network (NN) with gradientdecoder architecture which detects positions of heartbeats.
Second stage is a classification NN which will classify heartbeats as wide or narrow.
Third stage is a boosting decision trees (GBDT) on top of NN features that incorporates patient-wise features.
arXiv Detail & Related papers (2020-11-17T16:15:49Z) - BiteNet: Bidirectional Temporal Encoder Network to Predict Medical
Outcomes [53.163089893876645]
We propose a novel self-attention mechanism that captures the contextual dependency and temporal relationships within a patient's healthcare journey.
An end-to-end bidirectional temporal encoder network (BiteNet) then learns representations of the patient's journeys.
We have evaluated the effectiveness of our methods on two supervised prediction and two unsupervised clustering tasks with a real-world EHR dataset.
arXiv Detail & Related papers (2020-09-24T00:42:36Z) - Semi-supervised Medical Image Classification with Relation-driven
Self-ensembling Model [71.80319052891817]
We present a relation-driven semi-supervised framework for medical image classification.
It exploits the unlabeled data by encouraging the prediction consistency of given input under perturbations.
Our method outperforms many state-of-the-art semi-supervised learning methods on both single-label and multi-label image classification scenarios.
arXiv Detail & Related papers (2020-05-15T06:57:54Z) - Deep Representation Learning of Electronic Health Records to Unlock
Patient Stratification at Scale [0.5498849973527224]
We present an unsupervised framework based on deep learning to process heterogeneous EHRs.
We derive patient representations that can efficiently and effectively enable patient stratification at scale.
arXiv Detail & Related papers (2020-03-14T00:04:20Z) - DeepEnroll: Patient-Trial Matching with Deep Embedding and Entailment
Prediction [67.91606509226132]
Clinical trials are essential for drug development but often suffer from expensive, inaccurate and insufficient patient recruitment.
DeepEnroll is a cross-modal inference learning model to jointly encode enrollment criteria (tabular data) into a shared latent space for matching inference.
arXiv Detail & Related papers (2020-01-22T17:51:25Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.