Cross-Spectral Dense Correspondence for Multimodal Spectral Medical Imaging

• URL: http://arxiv.org/abs/2608.28341v1
• Date: Fri, 28 Aug 2026 13:53:29 GMT
• Title: Cross-Spectral Dense Correspondence for Multimodal Spectral Medical Imaging
• Abstract summary: We introduce a sensor-agnostic cross-spectral modulation protocol on established correspondence benchmarks with intensity input projection.<n>We show substantial improvements under severe spectral mismatch while maintaining performance on standard RGB benchmarks.
• Score: 9.00893522009341
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Precise dense correspondence is a fundamental prerequisite for multimodal spectral imaging systems that fuse disparate wavelength ranges for subsequent analysis in medical and scientific imaging. Corresponding image points are often observed with non-overlapping spectral sensitivities, leading to wavelength-dependent contrast changes, intensity inversions, and appearance shifts for which dense ground truth is difficult to obtain and conventional RGB-based training data provides only limited supervision. We address this data gap by introducing a sensor-agnostic cross-spectral modulation protocol on established correspondence benchmarks with intensity input projection, and by proposing a synthetic cross-spectral correspondence benchmark simulating physically plausible radiometric differences. Evaluation on several modern dense correspondence backbones trained with our unified cross-spectral protocol showed substantial improvements under severe spectral mismatch while maintaining performance on standard RGB benchmarks. Ablation experiments show that view-dependent channel selection and nonlinear radiometric transformations provide complementary robustness, indicating that the primary limitation of existing models is not their structural matching capacity but the mismatch between training distribution and spectral characteristics of the target image pair. Qualitative evaluations on heterogeneous medical spectral acquisition systems demonstrate the practical relevance of the proposed training data augmentation protocol as an enabler for spatially coherent spectral fusion in HSI workflows.

Related papers

• SSFT: A Lightweight Spectral-Spatial Fusion Transformer for Generic Hyperspectral Classification [42.09644109233563]
  We propose the lightweight Spectral-Spatial Fusion Transformer (SSFT), which factorizes representation learning into spectral and spatial pathways.<n>We evaluate our SSFT on the challenging HSI-Benchmark, a heterogeneous multi-dataset benchmark covering earth observation, fruit condition assessment, and fine-grained material recognition.
  arXiv  Detail & Related papers  (2026-04-17T08:31:25Z)
• Particle Diffusion Matching: Random Walk Correspondence Search for the Alignment of Standard and Ultra-Widefield Fundus Images [47.944896477309555]
  We propose a robust alignment technique for Standard Fundus Images (SFIs) and Ultra-Widefield Fundus Images (UWFIs)<n>Our method, termed Particle Diffusion Matching (PDM), performs alignment through an iterative Random Walk Correspondence Search (RWCS) guided by a diffusion model.<n>PDM achieves state-of-the-art performance across multiple retinal image alignment benchmarks.
  arXiv  Detail & Related papers  (2026-04-11T08:09:53Z)
• Spectral Discrepancy and Cross-modal Semantic Consistency Learning for Object Detection in Hyperspectral Image [40.38555448650773]
  Hyperspectral images with high spectral resolution provide new insights into recognizing subtle differences in similar substances.<n> object detection in hyperspectral images faces significant challenges in intra- and inter-class similarity due to the spatial differences in hyperspectral inter-bands.<n>We propose a novel network termed textbfSpectral textbfDiscrepancy and textbfCross-textbfModal semantic consistency learning (SDCM)<n>Our proposed method achieves state-of-the-art performance when compared with other ones.
  arXiv  Detail & Related papers  (2025-12-20T07:03:09Z)
• Uncertainty Quantification in HSI Reconstruction using Physics-Aware Diffusion Priors and Optics-Encoded Measurements [12.90606332269204]
  We formulate hyperspectral image (HSI) reconstruction as a Bayesian inference problem.<n>We propose a framework, HSDiff, that utilizes an unconditionally trained, pixel-level diffusion prior and posterior diffusion sampling.<n>We demonstrate that HSDiff offers a complete, high-performance method for uncertainty-aware HSI reconstruction.
  arXiv  Detail & Related papers  (2025-11-23T14:43:51Z)
• SpectralAdapt: Semi-Supervised Domain Adaptation with Spectral Priors for Human-Centered Hyperspectral Image Reconstruction [20.98412186634439]
  Hyperspectral imaging holds great potential for healthcare due to its rich spectral information.<n>However, acquiring HSI data remains costly and technically demanding.<n>We propose SpectralAdapt, a framework that bridges the domain gap between general and human-centered HSI datasets.
  arXiv  Detail & Related papers  (2025-11-17T06:17:13Z)
• CARL: Camera-Agnostic Representation Learning for Spectral Image Analysis [69.02751635551724]
  Spectral imaging offers promising applications across diverse domains, including medicine and urban scene understanding.<n> variability in channel dimensionality and captured wavelengths among spectral cameras impede the development of AI-driven methodologies.<n>We introduce CARL, a model for Camera-Agnostic Representation Learning across RGB, multispectral, and hyperspectral imaging modalities.
  arXiv  Detail & Related papers  (2025-04-27T13:06:40Z)
• ESSAformer: Efficient Transformer for Hyperspectral Image Super-resolution [76.7408734079706]
  Single hyperspectral image super-resolution (single-HSI-SR) aims to restore a high-resolution hyperspectral image from a low-resolution observation. We propose ESSAformer, an ESSA attention-embedded Transformer network for single-HSI-SR with an iterative refining structure.
  arXiv  Detail & Related papers  (2023-07-26T07:45:14Z)
• Exploring Invariant Representation for Visible-Infrared Person Re-Identification [77.06940947765406]
  Cross-spectral person re-identification, which aims to associate identities to pedestrians across different spectra, faces a main challenge of the modality discrepancy. In this paper, we address the problem from both image-level and feature-level in an end-to-end hybrid learning framework named robust feature mining network (RFM) Experiment results on two standard cross-spectral person re-identification datasets, RegDB and SYSU-MM01, have demonstrated state-of-the-art performance.
  arXiv  Detail & Related papers  (2023-02-02T05:24:50Z)
• Negligible effect of brain MRI data preprocessing for tumor segmentation [36.89606202543839]
  We conduct experiments on three publicly available datasets and evaluate the effect of different preprocessing steps in deep neural networks. Our results demonstrate that most popular standardization steps add no value to the network performance. We suggest that image intensity normalization approaches do not contribute to model accuracy because of the reduction of signal variance with image standardization.
  arXiv  Detail & Related papers  (2022-04-11T17:29:36Z)
• Cross-Spectral Periocular Recognition with Conditional Adversarial Networks [59.17685450892182]
  We propose Conditional Generative Adversarial Networks, trained to con-vert periocular images between visible and near-infrared spectra. We obtain a cross-spectral periocular performance of EER=1%, and GAR>99% @ FAR=1%, which is comparable to the state-of-the-art with the PolyU database.
  arXiv  Detail & Related papers  (2020-08-26T15:02:04Z)
• Hyperspectral-Multispectral Image Fusion with Weighted LASSO [68.04032419397677]
  We propose an approach for fusing hyperspectral and multispectral images to provide high-quality hyperspectral output. We demonstrate that the proposed sparse fusion and reconstruction provides quantitatively superior results when compared to existing methods on publicly available images.
  arXiv  Detail & Related papers  (2020-03-15T23:07:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.