Semi-Supervised Flow Matching for Mosaiced and Panchromatic Fusion Imaging

• URL: http://arxiv.org/abs/2604.20128v1
• Date: Wed, 22 Apr 2026 02:48:48 GMT
• Title: Semi-Supervised Flow Matching for Mosaiced and Panchromatic Fusion Imaging
• Abstract summary: We propose a novel semi-supervised flow matching framework for mosaiced and PAN image fusion.<n>Our method seamlessly integrates an unsupervised scheme with flow matching, resulting in a generalizable and efficient generative framework.
• Score: 16.83673519796277
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Fusing a low resolution (LR) mosaiced hyperspectral image (HSI) with a high resolution (HR) panchromatic (PAN) image offers a promising avenue for video-rate HR-HSI imaging via single-shot acquisition, yet its severely ill-posed nature remains a significant challenge. In this work, we propose a novel semi-supervised flow matching framework for mosaiced and PAN image fusion. Unlike previous diffusion-based approaches constrained by specific protocols or handcrafted assumptions, our method seamlessly integrates an unsupervised scheme with flow matching, resulting in a generalizable and efficient generative framework. Specifically, our method follows a two-stage training pipeline. First, we pretrain an unsupervised prior network to produce an initial pseudo HR-HSI. Building on this, we then train a conditional flow matching model to generate the target HR-HSI, introducing a random voting mechanism that iteratively refines the initial HR-HSI estimate, enabling robust and effective fusion. During inference, we employ a conflict-free gradient guidance strategy that ensures spectrally and spatially consistent HR-HSI reconstruction. Experiments on multiple benchmark datasets demonstrate that our method achieves superior quantitative and qualitative performance by a significant margin compared to representative baselines. Beyond mosaiced and PAN fusion, our approach provides a flexible generative framework that can be readily extended to other image fusion tasks and integrated with unsupervised or blind image restoration algorithms.

Related papers

• UniGRPO: Unified Policy Optimization for Reasoning-Driven Visual Generation [51.41441081823758]
  Unified models capable of interleaved generation have emerged as a promising paradigm.<n>We propose a unified reinforcement learning framework tailored for interleaved generation.<n>Our experiments demonstrate that this unified training recipe significantly enhances image generation quality through reasoning.
  arXiv  Detail & Related papers  (2026-03-24T17:59:17Z)
• Annealed Langevin Posterior Sampling (ALPS): A Rapid Algorithm for Image Restoration with Multiscale Energy Models [6.5812718077838]
  Energy-Based Models (EBMs) have historically suffered from high computational costs and training instability.<n>We propose Annealed Langevin Posterior Sampling (ALPS) algorithm for Maximum-A-Posteriori (MAP), Minimum Mean Square Error (MMSE) and uncertainty estimates for inverse problems in imaging.
  arXiv  Detail & Related papers  (2026-01-05T22:53:23Z)
• Unsupervised Hyperspectral and Multispectral Image Fusion via Self-Supervised Modality Decoupling [16.923899489263945]
  Hyperspectral and Multispectral Image Fusion (HMIF) aims to fuse low-resolution hyperspectral images (LR-HSIs) and high-resolution multispectral images (HR-MSIs)<n>HMIF methods typically apply direct fusion from the two modalities without effective supervision.<n>We propose an end-to-end self-supervised textbfModality-Decoupled textbfSpatial-textbfSpectral Fusion (textbfMossFuse) framework.
  arXiv  Detail & Related papers  (2024-12-06T06:22:43Z)
• Hipandas: Hyperspectral Image Joint Denoising and Super-Resolution by Image Fusion with the Panchromatic Image [51.333064033152304]
  Recently launched satellites can concurrently acquire HSIs and panchromatic (PAN) images.<n>Hipandas is a novel learning paradigm that reconstructs HRHS images from noisy low-resolution HSIs and high-resolution PAN images.
  arXiv  Detail & Related papers  (2024-12-05T14:39:29Z)
• Multi-Head Attention Residual Unfolded Network for Model-Based Pansharpening [2.874893537471256]
  Unfolding fusion methods integrate the powerful representation capabilities of deep learning with the robustness of model-based approaches. In this paper, we propose a model-based deep unfolded method for satellite image fusion. Experimental results on PRISMA, Quickbird, and WorldView2 datasets demonstrate the superior performance of our method.
  arXiv  Detail & Related papers  (2024-09-04T13:05:00Z)
• Timestep-Aware Diffusion Model for Extreme Image Rescaling [47.89362819768323]
  We propose a novel framework called Timestep-Aware Diffusion Model (TADM) for extreme image rescaling.<n>TADM performs rescaling operations in the latent space of a pre-trained autoencoder.<n>It effectively leverages powerful natural image priors learned by a pre-trained text-to-image diffusion model.
  arXiv  Detail & Related papers  (2024-08-17T09:51:42Z)
• Variational Zero-shot Multispectral Pansharpening [43.881891055500496]
  Pansharpening aims to generate a high spatial resolution multispectral image (HRMS) by fusing a low spatial resolution multispectral image (LRMS) and a panchromatic image (PAN) Existing deep learning-based methods are unsuitable since they rely on many training pairs. We propose a zero-shot pansharpening method by introducing a neural network into the optimization objective.
  arXiv  Detail & Related papers  (2024-07-09T07:59:34Z)
• JoReS-Diff: Joint Retinex and Semantic Priors in Diffusion Model for Low-light Image Enhancement [69.6035373784027]
  Low-light image enhancement (LLIE) has achieved promising performance by employing conditional diffusion models. Previous methods may neglect the importance of a sufficient formulation of task-specific condition strategy. We propose JoReS-Diff, a novel approach that incorporates Retinex- and semantic-based priors as the additional pre-processing condition.
  arXiv  Detail & Related papers  (2023-12-20T08:05:57Z)
• DDFM: Denoising Diffusion Model for Multi-Modality Image Fusion [144.9653045465908]
  We propose a novel fusion algorithm based on the denoising diffusion probabilistic model (DDPM) Our approach yields promising fusion results in infrared-visible image fusion and medical image fusion.
  arXiv  Detail & Related papers  (2023-03-13T04:06:42Z)
• Memory-augmented Deep Unfolding Network for Guided Image Super-resolution [67.83489239124557]
  Guided image super-resolution (GISR) aims to obtain a high-resolution (HR) target image by enhancing the spatial resolution of a low-resolution (LR) target image under the guidance of a HR image. Previous model-based methods mainly takes the entire image as a whole, and assume the prior distribution between the HR target image and the HR guidance image. We propose a maximal a posterior (MAP) estimation model for GISR with two types of prior on the HR target image.
  arXiv  Detail & Related papers  (2022-02-12T15:37:13Z)
• PGMAN: An Unsupervised Generative Multi-adversarial Network for Pan-sharpening [46.84573725116611]
  We propose an unsupervised framework that learns directly from the full-resolution images without any preprocessing. We use a two-stream generator to extract the modality-specific features from the PAN and MS images, respectively, and develop a dual-discriminator to preserve the spectral and spatial information of the inputs when performing fusion.
  arXiv  Detail & Related papers  (2020-12-16T16:21:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.