Deep Learning-Based Correction and Unmixing of Hyperspectral Images for Brain Tumor Surgery

• URL: http://arxiv.org/abs/2402.03761v1
• Date: Tue, 6 Feb 2024 07:04:35 GMT
• Title: Deep Learning-Based Correction and Unmixing of Hyperspectral Images for Brain Tumor Surgery
• Authors: David Black, Jaidev Gill, Andrew Xie, Benoit Liquet, Antonio Di leva, Walter Stummer, Eric Suero Molina
• Abstract summary: We propose two deep learning models for correction and unmixing. One is trained with protoporphyrin IX (PpIX) concentration labels. The other undergoes semi-supervised training, first learning hyperspectral unmixing self-supervised and then learning to correct fluorescence emission spectra.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Hyperspectral Imaging (HSI) for fluorescence-guided brain tumor resection enables visualization of differences between tissues that are not distinguishable to humans. This augmentation can maximize brain tumor resection, improving patient outcomes. However, much of the processing in HSI uses simplified linear methods that are unable to capture the non-linear, wavelength-dependent phenomena that must be modeled for accurate recovery of fluorophore abundances. We therefore propose two deep learning models for correction and unmixing, which can account for the nonlinear effects and produce more accurate estimates of abundances. Both models use an autoencoder-like architecture to process the captured spectra. One is trained with protoporphyrin IX (PpIX) concentration labels. The other undergoes semi-supervised training, first learning hyperspectral unmixing self-supervised and then learning to correct fluorescence emission spectra for heterogeneous optical and geometric properties using a reference white-light reflectance spectrum in a few-shot manner. The models were evaluated against phantom and pig brain data with known PpIX concentration; the supervised model achieved Pearson correlation coefficients (R values) between the known and computed PpIX concentrations of 0.997 and 0.990, respectively, whereas the classical approach achieved only 0.93 and 0.82. The semi-supervised approach's R values were 0.98 and 0.91, respectively. On human data, the semi-supervised model gives qualitatively more realistic results than the classical method, better removing bright spots of specular reflectance and reducing the variance in PpIX abundance over biopsies that should be relatively homogeneous. These results show promise for using deep learning to improve HSI in fluorescence-guided neurosurgery.

Related papers

• A Hybrid Deep Learning Classification of Perimetric Glaucoma Using Peripapillary Nerve Fiber Layer Reflectance and Other OCT Parameters from Three Anatomy Regions [0.907773831569615]
  A hybrid deep learning model combines NFL reflectance and other OCT parameters to improve glaucoma diagnosis. The overall accuracy was 0.948 with 0.893 sensitivity and 1.000 specificity, and the AROC was 0.979, which is significantly better than the logistic regression models.
  arXiv  Detail & Related papers  (2024-06-06T00:30:51Z)
• Gadolinium dose reduction for brain MRI using conditional deep learning [66.99830668082234]
  Two main challenges for these approaches are the accurate prediction of contrast enhancement and the synthesis of realistic images. We address both challenges by utilizing the contrast signal encoded in the subtraction images of pre-contrast and post-contrast image pairs. We demonstrate the effectiveness of our approach on synthetic and real datasets using various scanners, field strengths, and contrast agents.
  arXiv  Detail & Related papers  (2024-03-06T08:35:29Z)
• MAF-Net: Multiple attention-guided fusion network for fundus vascular image segmentation [1.3295074739915493]
  We propose a multiple attention-guided fusion network (MAF-Net) to accurately detect blood vessels in retinal fundus images. Traditional UNet-based models may lose partial information due to explicitly modeling long-distance dependencies. We show that our method produces satisfactory results compared to some state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-05-05T15:22:20Z)
• Early Diagnosis of Retinal Blood Vessel Damage via Deep Learning-Powered Collective Intelligence Models [0.3670422696827525]
  The power of swarm algorithms is used to search for various combinations of convolutional, pooling, and normalization layers to provide the best model for the task. The best TDCN model achieves an accuracy of 90.3%, AUC ROC of 0.956, and a Cohen score of 0.967.
  arXiv  Detail & Related papers  (2022-10-17T21:38:38Z)
• Multi-Channel Convolutional Analysis Operator Learning for Dual-Energy CT Reconstruction [108.06731611196291]
  We develop a multi-channel convolutional analysis operator learning (MCAOL) method to exploit common spatial features within attenuation images at different energies. We propose an optimization method which jointly reconstructs the attenuation images at low and high energies with a mixed norm regularization on the sparse features.
  arXiv  Detail & Related papers  (2022-03-10T14:22:54Z)
• Pseudo-Spherical Contrastive Divergence [119.28384561517292]
  We propose pseudo-spherical contrastive divergence (PS-CD) to generalize maximum learning likelihood of energy-based models. PS-CD avoids the intractable partition function and provides a generalized family of learning objectives.
  arXiv  Detail & Related papers  (2021-11-01T09:17:15Z)
• DeepSMILE: Contrastive self-supervised pre-training benefits MSI and HRD classification directly from H&E whole-slide images in colorectal and breast cancer [22.46523830554047]
  We propose a Deep learning-based weak label learning method for analyzing whole slide images (WSIs) of Hematoxylin and Eosin stained tumor tissue. We apply DeepSMILE to the task of Homologous recombination deficiency (HRD) and microsatellite instability (MSI) prediction.
  arXiv  Detail & Related papers  (2021-07-20T11:00:16Z)
• Prediction of low-keV monochromatic images from polyenergetic CT scans for improved automatic detection of pulmonary embolism [21.47219330040151]
  We are training convolutional neural networks that can emulate the generation of monoE images from conventional single energy CT acquisitions. We expand on these methods through the use of a multi-task optimization approach, under which the networks achieve improved classification as well as generation results.
  arXiv  Detail & Related papers  (2021-02-02T11:42:31Z)
• Classification of COVID-19 in CT Scans using Multi-Source Transfer Learning [91.3755431537592]
  We propose the use of Multi-Source Transfer Learning to improve upon traditional Transfer Learning for the classification of COVID-19 from CT scans. With our multi-source fine-tuning approach, our models outperformed baseline models fine-tuned with ImageNet. Our best performing model was able to achieve an accuracy of 0.893 and a Recall score of 0.897, outperforming its baseline Recall score by 9.3%.
  arXiv  Detail & Related papers  (2020-09-22T11:53:06Z)
• Improved Slice-wise Tumour Detection in Brain MRIs by Computing Dissimilarities between Latent Representations [68.8204255655161]
  Anomaly detection for Magnetic Resonance Images (MRIs) can be solved with unsupervised methods. We have proposed a slice-wise semi-supervised method for tumour detection based on the computation of a dissimilarity function in the latent space of a Variational AutoEncoder. We show that by training the models on higher resolution images and by improving the quality of the reconstructions, we obtain results which are comparable with different baselines.
  arXiv  Detail & Related papers  (2020-07-24T14:02:09Z)
• Data Augmentation for Histopathological Images Based on Gaussian-Laplacian Pyramid Blending [59.91656519028334]
  Data imbalance is a major problem that affects several machine learning (ML) algorithms. In this paper, we propose a novel approach capable of not only augmenting HI dataset but also distributing the inter-patient variability. Experimental results on the BreakHis dataset have shown promising gains vis-a-vis the majority of DA techniques presented in the literature.
  arXiv  Detail & Related papers  (2020-01-31T22:02:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.