A CNN-Transformer for Classification of Longitudinal 3D MRI Images -- A Case Study on Hepatocellular Carcinoma Prediction

• URL: http://arxiv.org/abs/2501.10733v2
• Date: Wed, 22 Jan 2025 10:50:37 GMT
• Title: A CNN-Transformer for Classification of Longitudinal 3D MRI Images -- A Case Study on Hepatocellular Carcinoma Prediction
• Authors: Jakob Nolte, Maureen M. J. Guichelaar, Donald E. Bouman, Stephanie M. van den Berg, Maryam Amir Haeri,
• Abstract summary: HCCNet is a novel model architecture that integrates a 3D adaptation of the ConvNeXt CNN architecture with a Transformer encoder. Our results show that HCCNet significantly improves predictive accuracy and reliability over baseline models.
• Score: 0.0
• License:
• Abstract: Longitudinal MRI analysis is crucial for predicting disease outcomes, particularly in chronic conditions like hepatocellular carcinoma (HCC), where early detection can significantly influence treatment strategies and patient prognosis. Yet, due to challenges like limited data availability, subtle parenchymal changes, and the irregular timing of medical screenings, current approaches have so far focused on cross-sectional imaging data. To address this, we propose HCCNet, a novel model architecture that integrates a 3D adaptation of the ConvNeXt CNN architecture with a Transformer encoder, capturing both the intricate spatial features of 3D MRIs and the complex temporal dependencies across different time points. HCCNet utilizes a two-stage pre-training process tailored for longitudinal MRI data. The CNN backbone is pre-trained using a self-supervised learning framework adapted for 3D MRIs, while the Transformer encoder is pre-trained with a sequence-order-prediction task to enhance its understanding of disease progression over time. We demonstrate the effectiveness of HCCNet by applying it to a cohort of liver cirrhosis patients undergoing regular MRI screenings for HCC surveillance. Our results show that HCCNet significantly improves predictive accuracy and reliability over baseline models, providing a robust tool for personalized HCC surveillance. The methodological approach presented in this paper is versatile and can be adapted to various longitudinal MRI screening applications. Its ability to handle varying patient record lengths and irregular screening intervals establishes it as an invaluable framework for monitoring chronic diseases, where timely and accurate disease prognosis is critical for effective treatment planning.

Related papers

• 4D VQ-GAN: Synthesising Medical Scans at Any Time Point for Personalised Disease Progression Modelling of Idiopathic Pulmonary Fibrosis [5.926086195644801]
  We propose 4D Vector Quantised Generative Adversarial Networks (4D-VQ-GAN), a model capable of generating realistic CT volumes of IPF patients. We evaluate different configurations of our model for generating longitudinal CT scans and compare the results against ground truth data.
  arXiv  Detail & Related papers  (2025-02-08T22:25:53Z)
• MPBD-LSTM: A Predictive Model for Colorectal Liver Metastases Using Time Series Multi-phase Contrast-Enhanced CT Scans [8.437136847051546]
  We build upon state-of-the-art deep learning techniques to evaluate how to best predict colorectal cancer liver metastasis. Our experimental results show that a multi-plane architecture based on 3D bi-directional LSTM, which we call MPBD-LSTM, works best.
  arXiv  Detail & Related papers  (2024-12-02T21:02:11Z)
• Probabilistic 3D Correspondence Prediction from Sparse Unsegmented Images [1.2179682412409507]
  We propose SPI-CorrNet, a unified model that predicts 3D correspondences from sparse imaging data. Experiments on the LGE MRI left atrium dataset and Abdomen CT-1K liver datasets demonstrate that our technique enhances the accuracy and robustness of sparse image-driven SSM.
  arXiv  Detail & Related papers  (2024-07-02T03:56:20Z)
• Enhancing MRI-Based Classification of Alzheimer's Disease with Explainable 3D Hybrid Compact Convolutional Transformers [13.743241062824548]
  Alzheimer's disease (AD) presents a formidable global health challenge. Traditional analysis methods often struggle to discern intricate 3D patterns crucial for AD identification. We introduce the 3D Hybrid Compact Convolutional Transformers 3D (HCCT)
  arXiv  Detail & Related papers  (2024-03-24T14:35:06Z)
• Swin-Tempo: Temporal-Aware Lung Nodule Detection in CT Scans as Video Sequences Using Swin Transformer-Enhanced UNet [2.7547288571938795]
  We present an innovative model that harnesses the strengths of both convolutional neural networks and vision transformers. Inspired by object detection in videos, we treat each 3D CT image as a video, individual slices as frames, and lung nodules as objects, enabling a time-series application.
  arXiv  Detail & Related papers  (2023-10-05T07:48:55Z)
• Automatic diagnosis of knee osteoarthritis severity using Swin transformer [55.01037422579516]
  Knee osteoarthritis (KOA) is a widespread condition that can cause chronic pain and stiffness in the knee joint. We propose an automated approach that employs the Swin Transformer to predict the severity of KOA.
  arXiv  Detail & Related papers  (2023-07-10T09:49:30Z)
• SurvLatent ODE : A Neural ODE based time-to-event model with competing risks for longitudinal data improves cancer-associated Deep Vein Thrombosis (DVT) prediction [68.8204255655161]
  We propose a generative time-to-event model, SurvLatent ODE, which parameterizes a latent representation under irregularly sampled data. Our model then utilizes the latent representation to flexibly estimate survival times for multiple competing events without specifying shapes of event-specific hazard function. SurvLatent ODE outperforms the current clinical standard Khorana Risk scores for stratifying DVT risk groups.
  arXiv  Detail & Related papers  (2022-04-20T17:28:08Z)
• Lung Cancer Lesion Detection in Histopathology Images Using Graph-Based Sparse PCA Network [93.22587316229954]
  We propose a graph-based sparse principal component analysis (GS-PCA) network, for automated detection of cancerous lesions on histological lung slides stained by hematoxylin and eosin (H&E) We evaluate the performance of the proposed algorithm on H&E slides obtained from an SVM K-rasG12D lung cancer mouse model using precision/recall rates, F-score, Tanimoto coefficient, and area under the curve (AUC) of the receiver operator characteristic (ROC)
  arXiv  Detail & Related papers  (2021-10-27T19:28:36Z)
• MIA-Prognosis: A Deep Learning Framework to Predict Therapy Response [58.0291320452122]
  This paper aims at a unified deep learning approach to predict patient prognosis and therapy response. We formalize the prognosis modeling as a multi-modal asynchronous time series classification task. Our predictive model could further stratify low-risk and high-risk patients in terms of long-term survival.
  arXiv  Detail & Related papers  (2020-10-08T15:30:17Z)
• Probabilistic 3D surface reconstruction from sparse MRI information [58.14653650521129]
  We present a novel probabilistic deep learning approach for concurrent 3D surface reconstruction from sparse 2D MR image data and aleatoric uncertainty prediction. Our method is capable of reconstructing large surface meshes from three quasi-orthogonal MR imaging slices from limited training sets.
  arXiv  Detail & Related papers  (2020-10-05T14:18:52Z)
• Spatio-spectral deep learning methods for in-vivo hyperspectral laryngeal cancer detection [49.32653090178743]
  Early detection of head and neck tumors is crucial for patient survival. Hyperspectral imaging (HSI) can be used for non-invasive detection of head and neck tumors. We present multiple deep learning techniques for in-vivo laryngeal cancer detection based on HSI.
  arXiv  Detail & Related papers  (2020-04-21T17:07:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.