Transfer learning RGB models to hyperspectral images with trainable tensor decompositions

• URL: http://arxiv.org/abs/2605.28331v1
• Date: Wed, 27 May 2026 11:34:27 GMT
• Title: Transfer learning RGB models to hyperspectral images with trainable tensor decompositions
• Abstract summary: Large vision networks assume the input images to have 3 input channels, making them incompatible with multi- or hyperspectral images.<n>This work proposes a novel approach that preserves the image and spatial information present in the model by using partially trainable tensor decompositions.<n> Experiments on a variety of hyperspectral datasets show that our approach is more accurate and robust than other hyperspectral transfer learning methods.
• Score: 6.71225143313941
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Transfer learning makes it possible to use large vision networks on a variety of domains, by specializing their models' general filters to new tasks. However, these networks assume the input images to have 3 input channels, making them incompatible with multi- or hyperspectral images. Current approaches that mitigate this incompatibility sacrifice information in either the image, or the model. This work proposes a novel approach that preserves the image and spatial information present in the model by using partially trainable tensor decompositions. We create such decompositions of pretrained convolutional filters, separating the filters into spatial and spectral components. The spectral components are then replaced with trainable components of higher channel dimensionality. This creates hyperspectral filters that can specialize to new datasets, while retaining the spatial patterns of the original filter. Experiments on a variety of hyperspectral datasets show that our approach is more accurate and robust than other hyperspectral transfer learning methods.

Related papers

• Freqformer: Image-Demoiréing Transformer via Efficient Frequency Decomposition [83.40450475728792]
  We present Freqformer, a Transformer-based framework specifically designed for image demoir'eing through targeted frequency separation.<n>Our method performs an effective frequency decomposition that explicitly splits moir'e patterns into high-frequency spatially-localized textures and low-frequency scale-robust color distortions.<n>Experiments on various demoir'eing benchmarks demonstrate that Freqformer achieves state-of-the-art performance with a compact model size.
  arXiv  Detail & Related papers  (2025-05-25T12:23:10Z)
• The Master Key Filters Hypothesis: Deep Filters Are General [51.900488744931785]
  Convolutional neural network (CNN) filters become increasingly specialized in deeper layers.<n>Recent observations of clusterable repeating patterns in depthwise separable CNNs (DS-CNNs) trained on ImageNet motivated this paper.<n>Our analysis of DS-CNNs reveals that deep filters maintain generality, contradicting the expected transition to class-specific filters.
  arXiv  Detail & Related papers  (2024-12-21T20:04:23Z)
• Spectral Image Data Fusion for Multisource Data Augmentation [44.99833362998488]
  Multispectral and hyperspectral images are increasingly popular in different research fields, such as remote sensing, astronomical imaging, or precision agriculture. The amount of free data available to perform machine learning tasks is relatively small. Artificial intelligence models developed in the area of spectral imaging require input images with a fixed spectral signature.
  arXiv  Detail & Related papers  (2024-04-05T13:40:18Z)
• Hyper-VolTran: Fast and Generalizable One-Shot Image to 3D Object Structure via HyperNetworks [53.67497327319569]
  We introduce a novel neural rendering technique to solve image-to-3D from a single view. Our approach employs the signed distance function as the surface representation and incorporates generalizable priors through geometry-encoding volumes and HyperNetworks. Our experiments show the advantages of our proposed approach with consistent results and rapid generation.
  arXiv  Detail & Related papers  (2023-12-24T08:42:37Z)
• Rotation-Scale Equivariant Steerable Filters [1.213915839836187]
  Digital histology imaging of biopsy tissue can be captured at arbitrary orientation and magnification and stored at different resolutions. We propose the Rotation-Scale Equivariant Steerable Filter (RSESF), which incorporates steerable filters and scale-space theory. Our method outperforms other approaches, with much fewer trainable parameters and fewer GPU resources required.
  arXiv  Detail & Related papers  (2023-04-10T14:13:56Z)
• Scale-Equivariant UNet for Histopathology Image Segmentation [1.213915839836187]
  Convolutional Neural Networks (CNNs) trained on such images at a given scale fail to generalise to those at different scales. We propose the Scale-Equivariant UNet (SEUNet) for image segmentation by building on scale-space theory.
  arXiv  Detail & Related papers  (2023-04-10T14:03:08Z)
• Multi-temporal speckle reduction with self-supervised deep neural networks [2.9979894869734927]
  Latest techniques rely on deep neural networks to restore the various structures and peculiar textures to SAR images. Speckle filtering is generally a prerequisite to the analysis of synthetic aperture radar (SAR) images. We extend a recent self-supervised training strategy for single-look complex SAR images, called MERLIN, to the case of multi-temporal filtering.
  arXiv  Detail & Related papers  (2022-07-22T14:08:22Z)
• SpectralFormer: Rethinking Hyperspectral Image Classification with Transformers [91.09957836250209]
  Hyperspectral (HS) images are characterized by approximately contiguous spectral information. CNNs have been proven to be a powerful feature extractor in HS image classification. We propose a novel backbone network called ulSpectralFormer for HS image classification.
  arXiv  Detail & Related papers  (2021-07-07T02:59:21Z)
• Global Filter Networks for Image Classification [90.81352483076323]
  We present a conceptually simple yet computationally efficient architecture that learns long-term spatial dependencies in the frequency domain with log-linear complexity. Our results demonstrate that GFNet can be a very competitive alternative to transformer-style models and CNNs in efficiency, generalization ability and robustness.
  arXiv  Detail & Related papers  (2021-07-01T17:58:16Z)
• Deeply Learned Spectral Total Variation Decomposition [8.679020335206753]
  We present a neural network approximation of a non-linear spectral decomposition. We report up to four orders of magnitude ($times 10,000$) speedup in processing of mega-pixel size images.
  arXiv  Detail & Related papers  (2020-06-17T17:10:43Z)
• Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral Imagery [79.69449412334188]
  In this paper, we investigate how to adapt state-of-the-art residual learning based single gray/RGB image super-resolution approaches. We introduce a spatial-spectral prior network (SSPN) to fully exploit the spatial information and the correlation between the spectra of the hyperspectral data. Experimental results on some hyperspectral images demonstrate that the proposed SSPSR method enhances the details of the recovered high-resolution hyperspectral images.
  arXiv  Detail & Related papers  (2020-05-18T14:25:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.