Dual Modality Prompted Diffusion Priors for Zero Shot Hyperspectral Pansharpening

• URL: http://arxiv.org/abs/2608.11748v2
• Date: Wed, 19 Aug 2026 06:54:34 GMT
• Title: Dual Modality Prompted Diffusion Priors for Zero Shot Hyperspectral Pansharpening
• Abstract summary: Hyperspectral pansharpening aims to reconstruct a high resolution hyperspectral (HRHS) image from a panchromatic (PAN) image and a low resolution hyperspectral (LRHS) image.<n>Recent diffusion based methods exploit pretrained image priors by generating a low dimensional representation and mapping it to the hyperspectral domain.<n>We propose dual-modality image-prompted diffusion model (DIDM) for zero shot hyperspectral pansharpening.
• Score: 21.84902245687311
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Hyperspectral pansharpening aims to reconstruct a high resolution hyperspectral (HRHS) image from a panchromatic (PAN) image and a low resolution hyperspectral (LRHS) image while preserving both spatial details and spectral fidelity. Recent diffusion based methods exploit pretrained image priors by generating a low dimensional representation and subsequently mapping it to the hyperspectral domain. However, the observed panchromatic and hyperspectral images are typically imposed only through external reconstruction objectives, limiting their direct interaction with the diffusion prior. To address this issue, we propose dual-modality image-prompted diffusion model (DIDM) for zero shot hyperspectral pansharpening. DIDM encodes the low resolution hyperspectral and panchromatic observations into spectral and spatial prompt tokens, respectively, and injects them into intermediate features of a frozen remote sensing diffusion model through cross attention, allowing complementary spectral and spatial information to directly guide diffusion feature evolution. In addition, we introduce a panchromatic guided weighted pixel aware total variation regularizer that combines low resolution hyperspectral degradation fidelity and panchromatic response fidelity with gradient adaptive structural regularization, thereby preserving structural discontinuities while suppressing spurious variations in homogeneous regions. Extensive experiments on Pavia, Chikusei, and Houston under reduced resolution protocols show that DIDM achieves the best performance across all evaluated metrics, while full resolution evaluation on FR1 yields the highest HQNR among the compared methods. These results demonstrate that internal dual modality prompting and panchromatic guided structural regularization provide an effective balance between spatial detail enhancement and spectral preservation.

Related papers

• AnyBand-Diff: A Unified Remote Sensing Image Generation and Band Repair Framework with Spectral Priors [10.352581041881864]
  This paper introduces AnyBand-Diff, a spectral-prior-guided diffusion framework tailored for robust spectral reconstruction.<n>To ensure radiometric fidelity, a Physics-Guided Sampling mechanism is proposed.<n>Experiments confirm the effectiveness of AnyBand-Diff in generating reliable imagery.
  arXiv  Detail & Related papers  (2026-05-14T04:04:34Z)
• ExtraVAR: Stage-Aware RoPE Remapping for Resolution Extrapolation in Visual Autoregressive Models [52.648413887350195]
  We propose Stage-Aware RoPE Remapping to suppress global repetition, local repetition, and detail degradation.<n>We also propose Entropy-Driven Adaptive Attention, which quantifies dispersion via a resolution-invariant normalized entropy.<n>Our method consistently outperforms prior resolution-extrapolation methods in both structural coherence and fine-detail fidelity.
  arXiv  Detail & Related papers  (2026-05-11T06:14:38Z)
• ASSR-Net: Anisotropic Structure-Aware and Spectrally Recalibrated Network for Hyperspectral Image Fusion [42.573141390377415]
  We propose textbfASSR-Net: an Anisotropic Structure-Aware and Spectrally Recalibrated Network for Hyperspectral Image Fusion.<n>ASSR-Net consistently outperforms state-of-the-art methods, achieving superior spatial detail preservation and spectral consistency.
  arXiv  Detail & Related papers  (2026-04-07T11:44:04Z)
• Universal Pansharpening Foundation Model [67.10467574892282]
  Pansharpening generates the high-resolution multi-spectral (MS) image by integrating spatial details from a texture-rich panchromatic (PAN) image and spectral attributes from a low-resolution MS image.<n>We present FoundPS, a universal pansharpening foundation model for satellite-agnostic and scene-robust fusion.
  arXiv  Detail & Related papers  (2026-03-04T08:30:15Z)
• SALAD-Pan: Sensor-Agnostic Latent Adaptive Diffusion for Pan-Sharpening [50.44337053599724]
  SALAD-Pan is a sensor-agnostic latent space diffusion method for efficient pansharpening.<n>It trains a band-wise single-channel VAE to encode high-resolution multispectral images into compact latent representations.<n>It achieves high-precision fusion in the diffusion process and robust zero-shot (cross-sensor) capability.
  arXiv  Detail & Related papers  (2026-02-04T12:01:07Z)
• Hyperspectral Image Generation with Unmixing Guided Diffusion Model [21.078386095749398]
  Hyperspectral image synthesis is a problem that has garnered growing interest yet remains constrained by the conditional generative paradigms that limit sample diversity.<n>We introduce a diffusion framework explicitly guided by hyperspectral unmixing to overcome the challenges.<n>The approach integrates two collaborative components: (i) an unmixing autoencoder that projects generation from the image domain into a low-dimensional abundance manifold, and (ii) an abundance diffusion process that enforces non-negativity and sum-to-one constraints.
  arXiv  Detail & Related papers  (2025-06-03T08:27:10Z)
• CARL: Camera-Agnostic Representation Learning for Spectral Image Analysis [69.02751635551724]
  Spectral imaging offers promising applications across diverse domains, including medicine and urban scene understanding.<n> variability in channel dimensionality and captured wavelengths among spectral cameras impede the development of AI-driven methodologies.<n>We introduce CARL, a model for Camera-Agnostic Representation Learning across RGB, multispectral, and hyperspectral imaging modalities.
  arXiv  Detail & Related papers  (2025-04-27T13:06:40Z)
• Physics-Inspired Degradation Models for Hyperspectral Image Fusion [61.743696362028246]
  Most fusion methods solely focus on the fusion algorithm itself and overlook the degradation models. We propose physics-inspired degradation models (PIDM) to model the degradation of LR-HSI and HR-MSI. Our proposed PIDM can boost the fusion performance of existing fusion methods in practical scenarios.
  arXiv  Detail & Related papers  (2024-02-04T09:07:28Z)
• PC-GANs: Progressive Compensation Generative Adversarial Networks for Pan-sharpening [50.943080184828524]
  We propose a novel two-step model for pan-sharpening that sharpens the MS image through the progressive compensation of the spatial and spectral information. The whole model is composed of triple GANs, and based on the specific architecture, a joint compensation loss function is designed to enable the triple GANs to be trained simultaneously.
  arXiv  Detail & Related papers  (2022-07-29T03:09:21Z)
• Learning Spatial-Spectral Prior for Super-Resolution of Hyperspectral Imagery [79.69449412334188]
  In this paper, we investigate how to adapt state-of-the-art residual learning based single gray/RGB image super-resolution approaches. We introduce a spatial-spectral prior network (SSPN) to fully exploit the spatial information and the correlation between the spectra of the hyperspectral data. Experimental results on some hyperspectral images demonstrate that the proposed SSPSR method enhances the details of the recovered high-resolution hyperspectral images.
  arXiv  Detail & Related papers  (2020-05-18T14:25:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.