Fose: Fusion of One-Step Diffusion and End-to-End Network for Pansharpening

• URL: http://arxiv.org/abs/2512.17202v1
• Date: Fri, 19 Dec 2025 03:28:39 GMT
• Title: Fose: Fusion of One-Step Diffusion and End-to-End Network for Pansharpening
• Authors: Kai Liu, Zeli Lin, Weibo Wang, Linghe Kong, Yulun Zhang,
• Abstract summary: We propose a novel four-stage training strategy to obtain a lightweight network Fose.<n>We perform one-step distillation on an enhanced SOTA DM for pansharping to compress the inference process from 50 steps to only 1 step.<n>We achieve a 7.42 speedup ratio compared to the baseline DM while achieving much better performance.
• Score: 38.98262335768959
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Pansharpening is a significant image fusion task that fuses low-resolution multispectral images (LRMSI) and high-resolution panchromatic images (PAN) to obtain high-resolution multispectral images (HRMSI). The development of the diffusion models (DM) and the end-to-end models (E2E model) has greatly improved the frontier of pansharping. DM takes the multi-step diffusion to obtain an accurate estimation of the residual between LRMSI and HRMSI. However, the multi-step process takes large computational power and is time-consuming. As for E2E models, their performance is still limited by the lack of prior and simple structure. In this paper, we propose a novel four-stage training strategy to obtain a lightweight network Fose, which fuses one-step DM and an E2E model. We perform one-step distillation on an enhanced SOTA DM for pansharping to compress the inference process from 50 steps to only 1 step. Then we fuse the E2E model with one-step DM with lightweight ensemble blocks. Comprehensive experiments are conducted to demonstrate the significant improvement of the proposed Fose on three commonly used benchmarks. Moreover, we achieve a 7.42 speedup ratio compared to the baseline DM while achieving much better performance. The code and model are released at https://github.com/Kai-Liu001/Fose.

Related papers

• TwinFlow: Realizing One-step Generation on Large Models with Self-adversarial Flows [25.487712175353035]
  We propose TwinFlow, a framework for training 1-step generative models.<n>Our method achieves a GenEval score of 0.83 in 1-NFE on text-to-image tasks.<n>Our approach matches the performance of the original 100-NFE model on GenEval and DPG-Bench benchmarks.
  arXiv  Detail & Related papers  (2025-12-03T07:45:46Z)
• Deep End-to-End Posterior ENergy (DEEPEN) for image recovery [12.218356507147583]
  Current end-to-end (E2E) and plug-and-play (MAP) image algorithms approximate the maximum a posteriori (MAP) estimate but cannot offer sampling from the posterior distribution.<n>By contrast, it is challenging for diffusion models to be trained in an E2E fashion.<n>This paper introduces a Deep End-to-End Posergy EN (DEE) framework, which enables MAP estimation as well as sampling.
  arXiv  Detail & Related papers  (2025-03-21T15:50:54Z)
• One Diffusion Step to Real-World Super-Resolution via Flow Trajectory Distillation [60.54811860967658]
  FluxSR is a novel one-step diffusion Real-ISR based on flow matching models.<n>First, we introduce Flow Trajectory Distillation (FTD) to distill a multi-step flow matching model into a one-step Real-ISR.<n>Second, to improve image realism and address high-frequency artifact issues in generated images, we propose TV-LPIPS as a perceptual loss.
  arXiv  Detail & Related papers  (2025-02-04T04:11:29Z)
• Accelerate High-Quality Diffusion Models with Inner Loop Feedback [50.00066451431194]
  Inner Loop Feedback (ILF) is a novel approach to accelerate diffusion models' inference.<n>ILF trains a lightweight module to predict future features in the denoising process.<n>ILF achieves strong performance for both class-to-image generation with diffusion transformer (DiT) and text-to-image generation with DiT-based PixArt-alpha and PixArt-sigma.
  arXiv  Detail & Related papers  (2025-01-22T18:59:58Z)
• Diffusion Models Are Innate One-Step Generators [2.3359837623080613]
  Diffusion Models (DMs) can generate remarkable high-quality results. DMs' layers are differentially activated at different time steps, leading to an inherent capability to generate images in a single step. Our method achieves the SOTA results on CIFAR-10, AFHQv2 64x64 (FID 1.23), FFHQ 64x64 (FID 0.85) and ImageNet 64x64 (FID 1.16) with great efficiency.
  arXiv  Detail & Related papers  (2024-05-31T11:14:12Z)
• One-Step Image Translation with Text-to-Image Models [35.0987002313882]
  We introduce a general method for adapting a single-step diffusion model to new tasks and domains through adversarial learning objectives. We consolidate various modules of the vanilla latent diffusion model into a single end-to-end generator network with small trainable weights. Our model CycleGAN-Turbo outperforms existing GAN-based and diffusion-based methods for various scene translation tasks.
  arXiv  Detail & Related papers  (2024-03-18T17:59:40Z)
• LDM-ISP: Enhancing Neural ISP for Low Light with Latent Diffusion Models [54.93010869546011]
  We propose to leverage the pre-trained latent diffusion model to perform the neural ISP for enhancing extremely low-light images.<n>Specifically, to tailor the pre-trained latent diffusion model to operate on the RAW domain, we train a set of lightweight taming modules.<n>We observe different roles of UNet denoising and decoder reconstruction in the latent diffusion model, which inspires us to decompose the low-light image enhancement task into latent-space low-frequency content generation and decoding-phase high-frequency detail maintenance.
  arXiv  Detail & Related papers  (2023-12-02T04:31:51Z)
• ToddlerDiffusion: Interactive Structured Image Generation with Cascaded Schrödinger Bridge [63.00793292863]
  ToddlerDiffusion is a novel approach to decomposing the complex task of RGB image generation into simpler, interpretable stages. Our method, termed ToddlerDiffusion, cascades modality-specific models, each responsible for generating an intermediate representation. ToddlerDiffusion consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2023-11-24T15:20:01Z)
• DiffI2I: Efficient Diffusion Model for Image-to-Image Translation [108.82579440308267]
  Diffusion Model (DM) has emerged as the SOTA approach for image synthesis. DM can't perform well on some image-to-image translation (I2I) tasks. DiffI2I comprises three key components: a compact I2I prior extraction network (CPEN), a dynamic I2I transformer (DI2Iformer) and a denoising network.
  arXiv  Detail & Related papers  (2023-08-26T05:18:23Z)
• SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Seconds [88.06788636008051]
  Text-to-image diffusion models can create stunning images from natural language descriptions that rival the work of professional artists and photographers. These models are large, with complex network architectures and tens of denoising iterations, making them computationally expensive and slow to run. We present a generic approach that unlocks running text-to-image diffusion models on mobile devices in less than $2$ seconds.
  arXiv  Detail & Related papers  (2023-06-01T17:59:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.