ST-NeRP: Spatial-Temporal Neural Representation Learning with Prior Embedding for Patient-specific Imaging Study

• URL: http://arxiv.org/abs/2410.19283v1
• Date: Fri, 25 Oct 2024 03:33:17 GMT
• Title: ST-NeRP: Spatial-Temporal Neural Representation Learning with Prior Embedding for Patient-specific Imaging Study
• Authors: Liang Qiu, Liyue Shen, Lianli Liu, Junyan Liu, Yizheng Chen, Lei Xing,
• Abstract summary: We propose a strategy of spatial-temporal Neural Representation learning with Prior embedding (ST-NeRP) for patient-specific imaging study. Our strategy involves leveraging an Implicit Neural Representation (INR) network to encode the image at the reference time point into a prior embedding. This network is trained using the whole patient-specific image sequence, enabling the prediction of deformation fields at various target time points.
• Score: 16.383405461343678
• License:
• Abstract: During and after a course of therapy, imaging is routinely used to monitor the disease progression and assess the treatment responses. Despite of its significance, reliably capturing and predicting the spatial-temporal anatomic changes from a sequence of patient-specific image series presents a considerable challenge. Thus, the development of a computational framework becomes highly desirable for a multitude of practical applications. In this context, we propose a strategy of Spatial-Temporal Neural Representation learning with Prior embedding (ST-NeRP) for patient-specific imaging study. Our strategy involves leveraging an Implicit Neural Representation (INR) network to encode the image at the reference time point into a prior embedding. Subsequently, a spatial-temporally continuous deformation function is learned through another INR network. This network is trained using the whole patient-specific image sequence, enabling the prediction of deformation fields at various target time points. The efficacy of the ST-NeRP model is demonstrated through its application to diverse sequential image series, including 4D CT and longitudinal CT datasets within thoracic and abdominal imaging. The proposed ST-NeRP model exhibits substantial potential in enabling the monitoring of anatomical changes within a patient throughout the therapeutic journey.

Related papers

• Unifying Subsampling Pattern Variations for Compressed Sensing MRI with Neural Operators [72.79532467687427]
  Compressed Sensing MRI reconstructs images of the body's internal anatomy from undersampled and compressed measurements. Deep neural networks have shown great potential for reconstructing high-quality images from highly undersampled measurements. We propose a unified model that is robust to different subsampling patterns and image resolutions in CS-MRI.
  arXiv  Detail & Related papers  (2024-10-05T20:03:57Z)
• Intraoperative Registration by Cross-Modal Inverse Neural Rendering [61.687068931599846]
  We present a novel approach for 3D/2D intraoperative registration during neurosurgery via cross-modal inverse neural rendering. Our approach separates implicit neural representation into two components, handling anatomical structure preoperatively and appearance intraoperatively. We tested our method on retrospective patients' data from clinical cases, showing that our method outperforms state-of-the-art while meeting current clinical standards for registration.
  arXiv  Detail & Related papers  (2024-09-18T13:40:59Z)
• Autoregressive Sequence Modeling for 3D Medical Image Representation [48.706230961589924]
  We introduce a pioneering method for learning 3D medical image representations through an autoregressive sequence pre-training framework. Our approach various 3D medical images based on spatial, contrast, and semantic correlations, treating them as interconnected visual tokens within a token sequence.
  arXiv  Detail & Related papers  (2024-09-13T10:19:10Z)
• Probabilistic 3D Correspondence Prediction from Sparse Unsegmented Images [1.2179682412409507]
  We propose SPI-CorrNet, a unified model that predicts 3D correspondences from sparse imaging data. Experiments on the LGE MRI left atrium dataset and Abdomen CT-1K liver datasets demonstrate that our technique enhances the accuracy and robustness of sparse image-driven SSM.
  arXiv  Detail & Related papers  (2024-07-02T03:56:20Z)
• ImageFlowNet: Forecasting Multiscale Image-Level Trajectories of Disease Progression with Irregularly-Sampled Longitudinal Medical Images [44.107186498384024]
  ImageFlowNet is a novel model designed to forecast disease trajectories from initial images while preserving spatial details. We provide theoretical insights that support our formulation of ODEs, and motivate our regularizations involving high-level visual features. Our contributions include the development of ImageFlowNet, its theoretical, and empirical validation on real-world datasets.
  arXiv  Detail & Related papers  (2024-06-20T23:51:32Z)
• CPT-Interp: Continuous sPatial and Temporal Motion Modeling for 4D Medical Image Interpolation [22.886841531680567]
  Motion information from 4D medical imaging offers critical insights into dynamic changes in patient anatomy for clinical assessments and radiotherapy planning. However, inherent physical and technical constraints of imaging hardware often necessitate a compromise between temporal resolution and image quality. We propose a novel approach for continuously modeling patient anatomic motion using implicit neural representation.
  arXiv  Detail & Related papers  (2024-05-24T09:35:42Z)
• CoNeS: Conditional neural fields with shift modulation for multi-sequence MRI translation [5.662694302758443]
  Multi-sequence magnetic resonance imaging (MRI) has found wide applications in both modern clinical studies and deep learning research. It frequently occurs that one or more of the MRI sequences are missing due to different image acquisition protocols or contrast agent contraindications of patients. One promising approach is to leverage generative models to synthesize the missing sequences, which can serve as a surrogate acquisition.
  arXiv  Detail & Related papers  (2023-09-06T19:01:58Z)
• Temporal Context Matters: Enhancing Single Image Prediction with Disease Progression Representations [8.396615243014768]
  We present a deep learning approach that leverages temporal progression information to improve clinical outcome predictions from single-timepoint images. In our method, a self-attention based Temporal Convolutional Network (TCN) is used to learn a representation that is most reflective of the disease trajectory. A Vision Transformer is pretrained in a self-supervised fashion to extract features from single-timepoint images.
  arXiv  Detail & Related papers  (2022-03-02T22:11:07Z)
• Learning Personal Representations from fMRIby Predicting Neurofeedback Performance [52.77024349608834]
  We present a deep neural network method for learning a personal representation for individuals performing a self neuromodulation task, guided by functional MRI (fMRI) The representation is learned by a self-supervised recurrent neural network, that predicts the Amygdala activity in the next fMRI frame given recent fMRI frames and is conditioned on the learned individual representation.
  arXiv  Detail & Related papers  (2021-12-06T10:16:54Z)
• Pathological Retinal Region Segmentation From OCT Images Using Geometric Relation Based Augmentation [84.7571086566595]
  We propose improvements over previous GAN-based medical image synthesis methods by jointly encoding the intrinsic relationship of geometry and shape. The proposed method outperforms state-of-the-art segmentation methods on the public RETOUCH dataset having images captured from different acquisition procedures.
  arXiv  Detail & Related papers  (2020-03-31T11:50:43Z)
• Learning Dynamic and Personalized Comorbidity Networks from Event Data using Deep Diffusion Processes [102.02672176520382]
  Comorbid diseases co-occur and progress via complex temporal patterns that vary among individuals. In electronic health records we can observe the different diseases a patient has, but can only infer the temporal relationship between each co-morbid condition. We develop deep diffusion processes to model "dynamic comorbidity networks"
  arXiv  Detail & Related papers  (2020-01-08T15:47:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.