Deep Learning for Cancer Prognosis Prediction Using Portrait Photos by StyleGAN Embedding

• URL: http://arxiv.org/abs/2306.14596v3
• Date: Thu, 25 Jul 2024 09:20:22 GMT
• Title: Deep Learning for Cancer Prognosis Prediction Using Portrait Photos by StyleGAN Embedding
• Authors: Amr Hagag, Ahmed Gomaa, Dominik Kornek, Andreas Maier, Rainer Fietkau, Christoph Bert, Florian Putz, Yixing Huang,
• Abstract summary: Survival prediction for cancer patients is critical for optimal treatment selection and patient management. Current patient survival prediction methods typically extract survival information from patients' clinical record data or biological and imaging data. In this work, the efficacy of objectively capturing and using prognostic information contained in conventional portrait photographs using deep learning for survival predication purposes is investigated.
• Score: 5.225384984555151
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Survival prediction for cancer patients is critical for optimal treatment selection and patient management. Current patient survival prediction methods typically extract survival information from patients' clinical record data or biological and imaging data. In practice, experienced clinicians can have a preliminary assessment of patients' health status based on patients' observable physical appearances, which are mainly facial features. However, such assessment is highly subjective. In this work, the efficacy of objectively capturing and using prognostic information contained in conventional portrait photographs using deep learning for survival predication purposes is investigated for the first time. A pre-trained StyleGAN2 model is fine-tuned on a custom dataset of our cancer patients' photos to empower its generator with generative ability suitable for patients' photos. The StyleGAN2 is then used to embed the photographs to its highly expressive latent space. Utilizing the state-of-the-art survival analysis models and based on StyleGAN's latent space photo embeddings, this approach achieved a C-index of 0.677, which is notably higher than chance and evidencing the prognostic value embedded in simple 2D facial images. In addition, thanks to StyleGAN's interpretable latent space, our survival prediction model can be validated for relying on essential facial features, eliminating any biases from extraneous information like clothing or background. Moreover, a health attribute is obtained from regression coefficients, which has important potential value for patient care.

Related papers

• Multi-task Learning Approach for Intracranial Hemorrhage Prognosis [0.0]
  We propose a 3D multi-task image model to predict prognosis, Glasgow Coma Scale and age, improving accuracy and interpretability. Our method outperforms current state-of-the-art baseline image models, and demonstrates superior performance in ICH prognosis compared to four board-certified neuroradiologists using only CT scans as input.
  arXiv  Detail & Related papers  (2024-08-16T14:56:17Z)
• SurvRNC: Learning Ordered Representations for Survival Prediction using Rank-N-Contrast [4.5445892770974154]
  Survival Rank-N Contrast (SurvRNC) is a loss function as a regularizer to obtain an ordered representation based on the survival times. We demonstrate that using the SurvRNC method for training can achieve higher performance on different deep survival models.
  arXiv  Detail & Related papers  (2024-03-15T18:00:11Z)
• OpticalDR: A Deep Optical Imaging Model for Privacy-Protective Depression Recognition [66.91236298878383]
  Depression Recognition (DR) poses a considerable challenge, especially in the context of privacy concerns. We design a new imaging system to erase the identity information of captured facial images while retain disease-relevant features. It is irreversible for identity information recovery while preserving essential disease-related characteristics necessary for accurate DR.
  arXiv  Detail & Related papers  (2024-02-29T01:20:29Z)
• Private, fair and accurate: Training large-scale, privacy-preserving AI models in medical imaging [47.99192239793597]
  We evaluated the effect of privacy-preserving training of AI models regarding accuracy and fairness compared to non-private training. Our study shows that -- under the challenging realistic circumstances of a real-life clinical dataset -- the privacy-preserving training of diagnostic deep learning models is possible with excellent diagnostic accuracy and fairness.
  arXiv  Detail & Related papers  (2023-02-03T09:49:13Z)
• Textual Data Augmentation for Patient Outcomes Prediction [67.72545656557858]
  We propose a novel data augmentation method to generate artificial clinical notes in patients' Electronic Health Records. We fine-tune the generative language model GPT-2 to synthesize labeled text with the original training data. We evaluate our method on the most common patient outcome, i.e., the 30-day readmission rate.
  arXiv  Detail & Related papers  (2022-11-13T01:07:23Z)
• Deterioration Prediction using Time-Series of Three Vital Signs and Current Clinical Features Amongst COVID-19 Patients [6.1594622252295474]
  We develop a prognostic model that predicts if a patient will experience deterioration in the forthcoming 3-24 hours. The model processes routine triadic vital signs: (a) oxygen saturation, (b) heart rate, and (c) temperature. We train and evaluate the model using data collected from 37,006 COVID-19 patients at NYU Langone Health in New York, USA.
  arXiv  Detail & Related papers  (2022-10-12T02:53:43Z)
• Predicting survival of glioblastoma from automatic whole-brain and tumor segmentation of MR images [1.0179233457605892]
  We introduce novel imaging features that can be automatically computed from MR images and fed into machine learning models to predict patient survival. The features measure the deformation caused by the tumor on the surrounding brain structures, comparing the shape of various structures in the patient's brain to their expected shape in healthy individuals. We show that the proposed features carry prognostic value in terms of overall- and progression-free survival, over and above that of conventional non-imaging features.
  arXiv  Detail & Related papers  (2021-09-25T10:49:51Z)
• Malignancy Prediction and Lesion Identification from Clinical Dermatological Images [65.1629311281062]
  We consider machine-learning-based malignancy prediction and lesion identification from clinical dermatological images. We first identify all lesions present in the image regardless of sub-type or likelihood of malignancy, then it estimates their likelihood of malignancy, and through aggregation, it also generates an image-level likelihood of malignancy.
  arXiv  Detail & Related papers  (2021-04-02T20:52:05Z)
• Variational Knowledge Distillation for Disease Classification in Chest X-Rays [102.04931207504173]
  We propose itvariational knowledge distillation (VKD), which is a new probabilistic inference framework for disease classification based on X-rays. We demonstrate the effectiveness of our method on three public benchmark datasets with paired X-ray images and EHRs.
  arXiv  Detail & Related papers  (2021-03-19T14:13:56Z)
• A Machine Learning Challenge for Prognostic Modelling in Head and Neck Cancer Using Multi-modal Data [0.10651507097431492]
  We have conducted an institutional machine learning challenge to develop an accurate model for overall survival prediction in head and neck cancer. We compared 12 different submissions using imaging and clinical data, separately or in combination. The winning approach used non-linear, multitask learning on clinical data and tumour volume, achieving high prognostic accuracy for 2-year and lifetime survival prediction.
  arXiv  Detail & Related papers  (2021-01-28T11:20:34Z)
• Integrative Analysis for COVID-19 Patient Outcome Prediction [53.11258640541513]
  We combine radiomics of lung opacities and non-imaging features from demographic data, vital signs, and laboratory findings to predict need for intensive care unit admission. Our methods may also be applied to other lung diseases including but not limited to community acquired pneumonia.
  arXiv  Detail & Related papers  (2020-07-20T19:08:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.