Leveraging Large-Scale Pretrained Spatial-Spectral Priors for General Zero-Shot Pansharpening

• URL: http://arxiv.org/abs/2512.02643v1
• Date: Tue, 02 Dec 2025 10:56:26 GMT
• Title: Leveraging Large-Scale Pretrained Spatial-Spectral Priors for General Zero-Shot Pansharpening
• Authors: Yongchuan Cui, Peng Liu, Yi Zeng,
• Abstract summary: Existing deep learning methods for remote sensing image fusion often suffer from poor generalization when applied to unseen datasets.<n>We propose a novel pretraining strategy that leverages large-scale simulated datasets to learn robust spatial-spectral priors.<n>The pretrained models achieve superior results in zero-shot scenarios and show remarkable adaptation capability with minimal real data in one-shot settings.
• Score: 7.37033839561317
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Existing deep learning methods for remote sensing image fusion often suffer from poor generalization when applied to unseen datasets due to the limited availability of real training data and the domain gap between different satellite sensors. To address this challenge, we explore the potential of foundation models by proposing a novel pretraining strategy that leverages large-scale simulated datasets to learn robust spatial-spectral priors. Specifically, our approach first constructs diverse simulated datasets by applying various degradation operations (blur, noise, downsampling) and augmentations (bands generation, channel shuffling, high-pass filtering, color jittering, etc.) to natural images from ImageNet and remote sensing images from SkyScript. We then pretrain fusion models on these simulated data to learn generalizable spatial-spectral representations. The pretrained models are subsequently evaluated on six datasets (WorldView-2/3/4, IKONOS, QuickBird, GaoFen-2) using zero-shot and one-shot paradigms, with both full- and freeze-tuning approaches for fine-tuning. Extensive experiments on different network architectures including convolutional neural networks, Transformer, and Mamba demonstrate that our pretraining strategy significantly improves generalization performance across different satellite sensors and imaging conditions for various fusion models. The pretrained models achieve superior results in zero-shot scenarios and show remarkable adaptation capability with minimal real data in one-shot settings. Our work provides a practical solution for cross-domain pansharpening, establishes a new benchmark for generalization in remote sensing image fusion tasks, and paves the way for leveraging foundation models through advanced training strategies.

Related papers

• Towards Realistic Remote Sensing Dataset Distillation with Discriminative Prototype-guided Diffusion [17.847157266396994]
  This study introduces the concept of dataset distillation into the field of remote sensing image interpretation.<n>We train a text-to-image diffusion model to condense a large-scale remote sensing dataset into a compact and representative distilled dataset.<n>Experiments on three high-resolution remote sensing scene classification benchmarks show that the proposed method can distill realistic and diverse samples for downstream model training.
  arXiv  Detail & Related papers  (2026-01-22T10:30:32Z)
• Dual-domain Adaptation Networks for Realistic Image Super-resolution [81.34345637776408]
  Realistic image super-resolution (SR) focuses on transforming real-world low-resolution (LR) images into high-resolution (HR) ones.<n>Current methods struggle with limited real-world LR-HR data, impacting the learning of basic image features.<n>We introduce a novel approach, which is able to efficiently adapt pre-trained image SR models from simulated to real-world datasets.
  arXiv  Detail & Related papers  (2025-11-21T12:57:23Z)
• FoundIR: Unleashing Million-scale Training Data to Advance Foundation Models for Image Restoration [66.61201445650323]
  Existing methods suffer from a generalization bottleneck in real-world scenarios.<n>We contribute a million-scale dataset with two notable advantages over existing training data.<n>We propose a robust model, FoundIR, to better address a broader range of restoration tasks in real-world scenarios.
  arXiv  Detail & Related papers  (2024-12-02T12:08:40Z)
• SatVision-TOA: A Geospatial Foundation Model for Coarse-Resolution All-Sky Remote Sensing Imagery [8.096413986108601]
  We introduce SatVision-TOA, a novel foundation model pre-trained on 14-band MODIS L1B Top-Of-Atmosphere (TOA) radiance imagery. The SatVision-TOA model is pre-trained using a Masked-Image-Modeling (MIM) framework and the SwinV2 architecture. Results show that SatVision-TOA achieves superior performance over baseline methods on downstream tasks.
  arXiv  Detail & Related papers  (2024-11-26T00:08:00Z)
• InfRS: Incremental Few-Shot Object Detection in Remote Sensing Images [11.916941756499435]
  In this paper, we explore the intricate task of incremental few-shot object detection in remote sensing images. We introduce a pioneering fine-tuning-based technique, termed InfRS, designed to facilitate the incremental learning of novel classes. We develop a prototypical calibration strategy based on the Wasserstein distance to mitigate the catastrophic forgetting problem.
  arXiv  Detail & Related papers  (2024-05-18T13:39:50Z)
• Deep Domain Adaptation: A Sim2Real Neural Approach for Improving Eye-Tracking Systems [80.62854148838359]
  Eye image segmentation is a critical step in eye tracking that has great influence over the final gaze estimate. We use dimensionality-reduction techniques to measure the overlap between the target eye images and synthetic training data. Our methods result in robust, improved performance when tackling the discrepancy between simulation and real-world data samples.
  arXiv  Detail & Related papers  (2024-03-23T22:32:06Z)
• Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery [78.43828998065071]
  Recent advances in unsupervised learning have demonstrated the ability of large vision models to achieve promising results on downstream tasks. Such pre-training techniques have also been explored recently in the remote sensing domain due to the availability of large amount of unlabelled data. In this paper, we re-visit transformers pre-training and leverage multi-scale information that is effectively utilized with multiple modalities.
  arXiv  Detail & Related papers  (2024-03-08T16:18:04Z)
• DiffusionSat: A Generative Foundation Model for Satellite Imagery [63.2807119794691]
  We present DiffusionSat, to date the largest generative foundation model trained on a collection of publicly available large, high-resolution remote sensing datasets. Our method produces realistic samples and can be used to solve multiple generative tasks including temporal generation, superresolution given multi-spectral inputs and in-painting.
  arXiv  Detail & Related papers  (2023-12-06T16:53:17Z)
• ExposureDiffusion: Learning to Expose for Low-light Image Enhancement [87.08496758469835]
  This work addresses the issue by seamlessly integrating a diffusion model with a physics-based exposure model. Our method obtains significantly improved performance and reduced inference time compared with vanilla diffusion models. The proposed framework can work with both real-paired datasets, SOTA noise models, and different backbone networks.
  arXiv  Detail & Related papers  (2023-07-15T04:48:35Z)
• Two-shot Spatially-varying BRDF and Shape Estimation [89.29020624201708]
  We propose a novel deep learning architecture with a stage-wise estimation of shape and SVBRDF. We create a large-scale synthetic training dataset with domain-randomized geometry and realistic materials. Experiments on both synthetic and real-world datasets show that our network trained on a synthetic dataset can generalize well to real-world images.
  arXiv  Detail & Related papers  (2020-04-01T12:56:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.