Explicit Temporal Embedding in Deep Generative Latent Models for Longitudinal Medical Image Synthesis

• URL: http://arxiv.org/abs/2301.05465v1
• Date: Fri, 13 Jan 2023 10:31:27 GMT
• Title: Explicit Temporal Embedding in Deep Generative Latent Models for Longitudinal Medical Image Synthesis
• Authors: Julian Sch\"on, Raghavendra Selvan, Lotte Nyg{\aa}rd, Ivan Richter Vogelius, Jens Petersen
• Abstract summary: We propose a novel joint learning scheme to embed temporal dependencies in the latent space of GANs. This allows us to synthesize continuous, smooth, and high-quality longitudinal volumetric data with limited supervision. We show the effectiveness of our approach on three datasets containing different longitudinal dependencies.
• Score: 1.1339580074756188
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Medical imaging plays a vital role in modern diagnostics and treatment. The temporal nature of disease or treatment progression often results in longitudinal data. Due to the cost and potential harm, acquiring large medical datasets necessary for deep learning can be difficult. Medical image synthesis could help mitigate this problem. However, until now, the availability of GANs capable of synthesizing longitudinal volumetric data has been limited. To address this, we use the recent advances in latent space-based image editing to propose a novel joint learning scheme to explicitly embed temporal dependencies in the latent space of GANs. This, in contrast to previous methods, allows us to synthesize continuous, smooth, and high-quality longitudinal volumetric data with limited supervision. We show the effectiveness of our approach on three datasets containing different longitudinal dependencies. Namely, modeling a simple image transformation, breathing motion, and tumor regression, all while showing minimal disentanglement. The implementation is made available online at https://github.com/julschoen/Temp-GAN.

Related papers

• Conditional Diffusion Model for Longitudinal Medical Image Generation [4.649475179575047]
  Alzheimers disease progresses slowly and involves complex interaction between biological factors. Longitudinal medical imaging data can capture this progression over time. However, longitudinal data frequently encounter issues such as missing data due to patient dropouts, irregular follow-up intervals, and varying lengths of observation periods.
  arXiv  Detail & Related papers  (2024-11-07T06:44:47Z)
• Longitudinal Causal Image Synthesis [19.07839779249869]
  Clinical decision-making relies heavily on causal reasoning and longitudinal analysis. How will the brain grey matter atrophy in a year if intervened on the A-beta level in cerebrospinal fluid?
  arXiv  Detail & Related papers  (2024-10-23T09:13:11Z)
• ImageFlowNet: Forecasting Multiscale Image-Level Trajectories of Disease Progression with Irregularly-Sampled Longitudinal Medical Images [44.107186498384024]
  ImageFlowNet is a novel model designed to forecast disease trajectories from initial images while preserving spatial details. We provide theoretical insights that support our formulation of ODEs, and motivate our regularizations involving high-level visual features. Our contributions include the development of ImageFlowNet, its theoretical, and empirical validation on real-world datasets.
  arXiv  Detail & Related papers  (2024-06-20T23:51:32Z)
• 3D MRI Synthesis with Slice-Based Latent Diffusion Models: Improving Tumor Segmentation Tasks in Data-Scarce Regimes [2.8498944632323755]
  We propose a novel slice-based latent diffusion architecture to address the complexities of volumetric data generation. This approach extends the joint distribution modeling of medical images and their associated masks, allowing a simultaneous generation of both under data-scarce regimes. Our architecture can be conditioned by tumor characteristics, including size, shape, and relative position, thereby providing a diverse range of tumor variations.
  arXiv  Detail & Related papers  (2024-06-08T09:53:45Z)
• Unlocking the Power of Spatial and Temporal Information in Medical Multimodal Pre-training [99.2891802841936]
  We introduce the Med-ST framework for fine-grained spatial and temporal modeling. For spatial modeling, Med-ST employs the Mixture of View Expert (MoVE) architecture to integrate different visual features from both frontal and lateral views. For temporal modeling, we propose a novel cross-modal bidirectional cycle consistency objective by forward mapping classification (FMC) and reverse mapping regression (RMR)
  arXiv  Detail & Related papers  (2024-05-30T03:15:09Z)
• CathFlow: Self-Supervised Segmentation of Catheters in Interventional Ultrasound Using Optical Flow and Transformers [66.15847237150909]
  We introduce a self-supervised deep learning architecture to segment catheters in longitudinal ultrasound images. The network architecture builds upon AiAReSeg, a segmentation transformer built with the Attention in Attention mechanism. We validated our model on a test dataset, consisting of unseen synthetic data and images collected from silicon aorta phantoms.
  arXiv  Detail & Related papers  (2024-03-21T15:13:36Z)
• SADM: Sequence-Aware Diffusion Model for Longitudinal Medical Image Generation [23.174391810184602]
  We propose a sequence-aware diffusion model (SADM) for the generation of longitudinal medical images. Our method extends this new technique by introducing a sequence-aware transformer as the conditional module in a diffusion model. Our experiments on 3D longitudinal medical images demonstrate the effectiveness of SADM compared with baselines and alternative methods.
  arXiv  Detail & Related papers  (2022-12-16T01:35:27Z)
• Image Synthesis with Disentangled Attributes for Chest X-Ray Nodule Augmentation and Detection [52.93342510469636]
  Lung nodule detection in chest X-ray (CXR) images is common to early screening of lung cancers. Deep-learning-based Computer-Assisted Diagnosis (CAD) systems can support radiologists for nodule screening in CXR. To alleviate the limited availability of such datasets, lung nodule synthesis methods are proposed for the sake of data augmentation.
  arXiv  Detail & Related papers  (2022-07-19T16:38:48Z)
• FetReg: Placental Vessel Segmentation and Registration in Fetoscopy Challenge Dataset [57.30136148318641]
  Fetoscopy laser photocoagulation is a widely used procedure for the treatment of Twin-to-Twin Transfusion Syndrome (TTTS) This may lead to increased procedural time and incomplete ablation, resulting in persistent TTTS. Computer-assisted intervention may help overcome these challenges by expanding the fetoscopic field of view through video mosaicking and providing better visualization of the vessel network. We present a large-scale multi-centre dataset for the development of generalized and robust semantic segmentation and video mosaicking algorithms for the fetal environment with a focus on creating drift-free mosaics from long duration fetoscopy videos.
  arXiv  Detail & Related papers  (2021-06-10T17:14:27Z)
• DeepRite: Deep Recurrent Inverse TreatmEnt Weighting for Adjusting Time-varying Confounding in Modern Longitudinal Observational Data [68.29870617697532]
  We propose Deep Recurrent Inverse TreatmEnt weighting (DeepRite) for time-varying confounding in longitudinal data. DeepRite is shown to recover the ground truth from synthetic data, and estimate unbiased treatment effects from real data.
  arXiv  Detail & Related papers  (2020-10-28T15:05:08Z)
• Learning Dynamic and Personalized Comorbidity Networks from Event Data using Deep Diffusion Processes [102.02672176520382]
  Comorbid diseases co-occur and progress via complex temporal patterns that vary among individuals. In electronic health records we can observe the different diseases a patient has, but can only infer the temporal relationship between each co-morbid condition. We develop deep diffusion processes to model "dynamic comorbidity networks"
  arXiv  Detail & Related papers  (2020-01-08T15:47:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.