S$^3$R: Self-supervised Spectral Regression for Hyperspectral Histopathology Image Classification

• URL: http://arxiv.org/abs/2209.08770v1
• Date: Mon, 19 Sep 2022 05:47:11 GMT
• Title: S$^3$R: Self-supervised Spectral Regression for Hyperspectral Histopathology Image Classification
• Authors: Xingran Xie, Yan Wang, and Qingli Li
• Abstract summary: We introduce an efficient and effective Self-supervised Spectral Regression (S$3$R) method, which exploits the low rank characteristic in the spectral domain of hyperspectral images (HSI) Two pre-text tasks are designed: (1)S$3$R-CR, which regresses the linear coefficients, so that the pre-trained model understands the inherent structures of HSIs and the pathological characteristics of different morphologies; (2)S$3$R-BR, which regresses the missing band, making the model to learn the holistic semantics of HSIs.
• Score: 13.388372624497807
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Benefited from the rich and detailed spectral information in hyperspectral images (HSI), HSI offers great potential for a wide variety of medical applications such as computational pathology. But, the lack of adequate annotated data and the high spatiospectral dimensions of HSIs usually make classification networks prone to overfit. Thus, learning a general representation which can be transferred to the downstream tasks is imperative. To our knowledge, no appropriate self-supervised pre-training method has been designed for histopathology HSIs. In this paper, we introduce an efficient and effective Self-supervised Spectral Regression (S$^3$R) method, which exploits the low rank characteristic in the spectral domain of HSI. More concretely, we propose to learn a set of linear coefficients that can be used to represent one band by the remaining bands via masking out these bands. Then, the band is restored by using the learned coefficients to reweight the remaining bands. Two pre-text tasks are designed: (1)S$^3$R-CR, which regresses the linear coefficients, so that the pre-trained model understands the inherent structures of HSIs and the pathological characteristics of different morphologies; (2)S$^3$R-BR, which regresses the missing band, making the model to learn the holistic semantics of HSIs. Compared to prior arts i.e., contrastive learning methods, which focuses on natural images, S$^3$R converges at least 3 times faster, and achieves significant improvements up to 14% in accuracy when transferring to HSI classification tasks.

Related papers

• Fine-tuning -- a Transfer Learning approach [0.22344294014777952]
  Missingness in Electronic Health Records (EHRs) is often hampered by the abundance of missing data in this valuable resource. Existing deep imputation methods rely on end-to-end pipelines that incorporate both imputation and downstream analyses. This paper explores the development of a modular, deep learning-based imputation and classification pipeline.
  arXiv  Detail & Related papers  (2024-11-06T14:18:23Z)
• HIR-Diff: Unsupervised Hyperspectral Image Restoration Via Improved Diffusion Models [38.74983301496911]
  Hyperspectral image (HSI) restoration aims at recovering clean images from degraded observations. Existing model-based methods have limitations in accurately modeling the complex image characteristics. This paper proposes an unsupervised HSI restoration framework with pre-trained diffusion model (HIR-Diff)
  arXiv  Detail & Related papers  (2024-02-24T17:15:05Z)
• Assessing Neural Network Representations During Training Using Noise-Resilient Diffusion Spectral Entropy [55.014926694758195]
  Entropy and mutual information in neural networks provide rich information on the learning process. We leverage data geometry to access the underlying manifold and reliably compute these information-theoretic measures. We show that they form noise-resistant measures of intrinsic dimensionality and relationship strength in high-dimensional simulated data.
  arXiv  Detail & Related papers  (2023-12-04T01:32:42Z)
• ESSAformer: Efficient Transformer for Hyperspectral Image Super-resolution [76.7408734079706]
  Single hyperspectral image super-resolution (single-HSI-SR) aims to restore a high-resolution hyperspectral image from a low-resolution observation. We propose ESSAformer, an ESSA attention-embedded Transformer network for single-HSI-SR with an iterative refining structure.
  arXiv  Detail & Related papers  (2023-07-26T07:45:14Z)
• Learning A 3D-CNN and Transformer Prior for Hyperspectral Image Super-Resolution [80.93870349019332]
  We propose a novel HSISR method that uses Transformer instead of CNN to learn the prior of HSIs. Specifically, we first use the gradient algorithm to solve the HSISR model, and then use an unfolding network to simulate the iterative solution processes.
  arXiv  Detail & Related papers  (2021-11-27T15:38:57Z)
• Cross-Site Severity Assessment of COVID-19 from CT Images via Domain Adaptation [64.59521853145368]
  Early and accurate severity assessment of Coronavirus disease 2019 (COVID-19) based on computed tomography (CT) images offers a great help to the estimation of intensive care unit event. To augment the labeled data and improve the generalization ability of the classification model, it is necessary to aggregate data from multiple sites. This task faces several challenges including class imbalance between mild and severe infections, domain distribution discrepancy between sites, and presence of heterogeneous features.
  arXiv  Detail & Related papers  (2021-09-08T07:56:51Z)
• Hyperspectral Image Super-Resolution with Spectral Mixup and Heterogeneous Datasets [99.92564298432387]
  This work studies Hyperspectral image (HSI) super-resolution (SR) HSI SR is characterized by high-dimensional data and a limited amount of training examples. This exacerbates the undesirable behaviors of neural networks such as memorization and sensitivity to out-of-distribution samples.
  arXiv  Detail & Related papers  (2021-01-19T12:19:53Z)
• DML-GANR: Deep Metric Learning With Generative Adversarial Network Regularization for High Spatial Resolution Remote Sensing Image Retrieval [9.423185775609426]
  We develop a deep metric learning approach with generative adversarial network regularization (DML-GANR) for HSR-RSI retrieval. The experimental results on the three data sets demonstrate the superior performance of DML-GANR over state-of-the-art techniques in HSR-RSI retrieval.
  arXiv  Detail & Related papers  (2020-10-07T02:26:03Z)
• Understanding Self-supervised Learning with Dual Deep Networks [74.92916579635336]
  We propose a novel framework to understand contrastive self-supervised learning (SSL) methods that employ dual pairs of deep ReLU networks. We prove that in each SGD update of SimCLR with various loss functions, the weights at each layer are updated by a emphcovariance operator. To further study what role the covariance operator plays and which features are learned in such a process, we model data generation and augmentation processes through a emphhierarchical latent tree model (HLTM)
  arXiv  Detail & Related papers  (2020-10-01T17:51:49Z)
• Improving Deep Hyperspectral Image Classification Performance with Spectral Unmixing [3.84448093764973]
  We propose an abundance-based multi-HSI classification method. We convert every HSI from the spectral domain to the abundance domain by a dataset-specific autoencoder. Secondly, the abundance representations from multiple HSIs are collected to form an enlarged dataset.
  arXiv  Detail & Related papers  (2020-04-01T17:14:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.