Fast Model-guided Instance-wise Adaptation Framework for Real-world Pansharpening with Fidelity Constraints

• URL: http://arxiv.org/abs/2604.08903v1
• Date: Fri, 10 Apr 2026 03:12:13 GMT
• Title: Fast Model-guided Instance-wise Adaptation Framework for Real-world Pansharpening with Fidelity Constraints
• Abstract summary: Pansharpening aims to generate high-resolution multispectral (HRMS) images by fusing low-resolution multispectral (LRMS) and high-resolution panchromatic (PAN) images.<n>Deep learning (DL)-based pansharpening methods achieve impressive performance, but require high training cost and large datasets.<n>Recent zero-shot methods, trained on a single PAN/LRMS pair, offer strong but suffer from limited fusion quality, high computational overhead, and slow convergence.
• Score: 23.251053308622186
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Pansharpening aims to generate high-resolution multispectral (HRMS) images by fusing low-resolution multispectral (LRMS) and high-resolution panchromatic (PAN) images while preserving both spectral and spatial information. Although deep learning (DL)-based pansharpening methods achieve impressive performance, they require high training cost and large datasets, and often degrade when the test distribution differs from training, limiting generalization. Recent zero-shot methods, trained on a single PAN/LRMS pair, offer strong generalization but suffer from limited fusion quality, high computational overhead, and slow convergence. To address these issues, we propose FMG-Pan, a fast and generalizable model-guided instance-wise adaptation framework for real-world pansharpening, achieving both cross-sensor generality and rapid training-inference. The framework leverages a pretrained model to guide a lightweight adaptive network through joint optimization with spectral and physical fidelity constraints. We further design a novel physical fidelity term to enhance spatial detail preservation. Extensive experiments on real-world datasets under both intra- and cross-sensor settings demonstrate state-of-the-art performance. On the WorldView-3 dataset, FMG-Pan completes training and inference for a 512x512x8 image within 3 seconds on an RTX 3090 GPU, significantly faster than existing zero-shot methods, making it suitable for practical deployment.

Related papers

• TIMI: Training-Free Image-to-3D Multi-Instance Generation with Spatial Fidelity [126.89088014048001]
  We propose a Training-free framework for Image-to-3D Multi-Instance generation that achieves high spatial fidelity.<n>Specifically, we first introduce an Instance-aware Separation Guidance (ISG) module, which facilitates instance disentanglement.<n>We then devise a Spatial-adaptive Geometry-stabilized Update (SGU) module that promotes the preservation of the geometric characteristics of instances.
  arXiv  Detail & Related papers  (2026-03-02T02:10:28Z)
• Scaling Text-to-Image Diffusion Transformers with Representation Autoencoders [74.72147962028265]
  Representation Autoencoders (RAEs) have shown distinct advantages in diffusion modeling on ImageNet.<n>We investigate whether this framework can scale to large-scale, freeform text-to-image (T2I) generation.
  arXiv  Detail & Related papers  (2026-01-22T18:58:16Z)
• API: Empowering Generalizable Real-World Image Dehazing via Adaptive Patch Importance Learning [12.516890497421203]
  We introduce a novel Adaptive Patch Importance-aware (API) framework for generalizable real-world image dehazing.<n>Specifically, our framework consists of an Automatic Haze Generation (AHG) module and a Density-aware Haze Removal (DHR) module.<n>To alleviate the ambiguity of the dehazed image details, we further introduce a new Multi-Negative Contrastive Dehazing (MNCD) loss.
  arXiv  Detail & Related papers  (2026-01-05T10:53:41Z)
• MuSASplat: Efficient Sparse-View 3D Gaussian Splats via Lightweight Multi-Scale Adaptation [92.57609195819647]
  MuSASplat is a novel framework that dramatically reduces the computational burden of training pose-free feed-forward 3D Gaussian splats models.<n>Central to our approach is a lightweight Multi-Scale Adapter that enables efficient fine-tuning of ViT-based architectures with only a small fraction of training parameters.
  arXiv  Detail & Related papers  (2025-12-08T04:56:46Z)
• SWIFT: A General Sensitive Weight Identification Framework for Fast Sensor-Transfer Pansharpening [16.578857961692716]
  Pansharpening aims to fuse high-resolution panchromatic (PAN) images with low-resolution multispectral (LRMS) images to generate high-resolution multispectral (HRMS) images.<n>Deep learning-based methods have achieved promising performance, but they generally suffer from severe performance degradation when applied to data from unseen sensors.<n>We propose a fast and general-purpose framework for cross-sensor adaptation, SWIFT.
  arXiv  Detail & Related papers  (2025-07-27T15:06:05Z)
• Improving Progressive Generation with Decomposable Flow Matching [50.63174319509629]
  Decomposable Flow Matching (DFM) is a simple and effective framework for the progressive generation of visual media.<n>On Imagenet-1k 512px, DFM achieves 35.2% improvements in FDD scores over the base architecture and 26.4% over the best-performing baseline.
  arXiv  Detail & Related papers  (2025-06-24T17:58:02Z)
• Two-Stage Random Alternation Framework for One-Shot Pansharpening [12.385955231193675]
  We introduce a two-stage random alternating framework (TRA-PAN) that performs instance-specific optimization for any given Multispectral(MS)/Panchromatic(PAN) pair.<n>TRA-PAN effectively integrates strong supervision constraints from reduced-resolution images with the physical characteristics of the full-resolution images.<n> Experimental results demonstrate that TRA-PAN outperforms state-of-the-art (SOTA) methods in quantitative metrics and visual quality in real-world scenarios.
  arXiv  Detail & Related papers  (2025-05-10T09:26:22Z)
• CAT: A Conditional Adaptation Tailor for Efficient and Effective Instance-Specific Pansharpening on Real-World Data [7.471505633354803]
  We propose an efficient framework that adapts to a specific input instance, completing both training and inference in a short time.<n>Our method achieves state-of-the-art performance on cross-sensor real-world data, while achieving both training and inference of $512times512$ image within $textit0.4 seconds$.
  arXiv  Detail & Related papers  (2025-04-14T14:04:55Z)
• Multi-Head Attention Residual Unfolded Network for Model-Based Pansharpening [2.874893537471256]
  Unfolding fusion methods integrate the powerful representation capabilities of deep learning with the robustness of model-based approaches. In this paper, we propose a model-based deep unfolded method for satellite image fusion. Experimental results on PRISMA, Quickbird, and WorldView2 datasets demonstrate the superior performance of our method.
  arXiv  Detail & Related papers  (2024-09-04T13:05:00Z)
• Efficient-3DiM: Learning a Generalizable Single-image Novel-view Synthesizer in One Day [63.96075838322437]
  We propose a framework to learn a single-image novel-view synthesizer. Our framework is able to reduce the total training time from 10 days to less than 1 day.
  arXiv  Detail & Related papers  (2023-10-04T17:57:07Z)
• Learning Deformable Image Registration from Optimization: Perspective, Modules, Bilevel Training and Beyond [62.730497582218284]
  We develop a new deep learning based framework to optimize a diffeomorphic model via multi-scale propagation. We conduct two groups of image registration experiments on 3D volume datasets including image-to-atlas registration on brain MRI data and image-to-image registration on liver CT data.
  arXiv  Detail & Related papers  (2020-04-30T03:23:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.