SGPDFuse: Semantically-Guided Physics-Disentanglement General Multi-Modal Image Fusion

• URL: http://arxiv.org/abs/2608.29220v1
• Date: Sat, 29 Aug 2026 12:08:27 GMT
• Title: SGPDFuse: Semantically-Guided Physics-Disentanglement General Multi-Modal Image Fusion
• Abstract summary: Multimodal image fusion aims to integrate complementary sensor data into a single representation that preserves intrinsic scene reality.<n>We propose SGPDFuse, which bridges the gap by mapping inputs into a physics-disentangled structural representation.<n>We show that SGPDFuse achieves state-of-the-art performance across infrared-visible, multi-focus, and multi-exposure benchmarks.
• Score: 12.582762404188257
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Multimodal image fusion (MMIF) aims to integrate complementary sensor data into a single representation that preserves intrinsic scene reality while eliminating environmental interferences. Most existing approaches rely on blind feature aggregation, which excels at signal accumulation but fails to distinguish essential content from physical degradations. We propose SGPDFuse, which bridges this gap by mapping inputs into a physics-disentangled structural representation via a Semantic-Physical Parametric Bridge (SPPB) built on pretrained vision foundation models, utilizing the Intrinsic-Variation principle to decouple invariant scene attributes from transient environmental factors. To guide this decomposition, we introduce a Semantic Alignment mechanism: we explicitly anchor the fused representation to salient semantic features in the same foundation model feature space via cosine similarity to preserve critical targets, while enforcing physical texture fidelity through Gram-matrix regularization to strictly eliminate unnatural artifacts. Extensive experiments demonstrate that SGPDFuse achieves state-of-the-art performance across infrared-visible, multi-focus, and multi-exposure benchmarks using a single architecture.

Related papers

• Proximity-CLIP: Text-Guided Semantic Proximity Learning for Zero-Shot Anomaly Detection [7.184018933441995]
  Vision-language models offer a promising approach for zero-shot anomaly detection (ZSAD)<n>Normal and anomalous text prototypes exhibit a high semantic overlap.<n>We propose Proximity-CLIP, a framework that visually calibrates the semantic margin to guide visual adaptation.
  arXiv  Detail & Related papers  (2026-09-07T08:48:27Z)
• Hierarchical Consistency Learning for Test-time Adaptation in Camouflage Perception [50.278200968044665]
  Camouflaged object detection (COD) aims to localize targets that exhibit minimal perceptual differences from backgrounds through physical attributes.<n>Existing methods, constrained by the static train-then-freeze paradigm, suffer from domain rigidity and dependency annotation.<n>We propose the hierarchical consistency learning framework, which integrates test-time adaptation for dynamic representation recalibration.
  arXiv  Detail & Related papers  (2026-05-25T09:57:46Z)
• RIDE: Retinex-Informed Decoupling for Exposing Concealed Objects [13.362481648112444]
  Concealed Object (COS) encompasses a family of dense-prediction tasks, including camouflaged object detection, polyp segmentation, transparent object detection, and industrial defect inspection.<n>Existing methods either operate directly on RGB images or employ emphheterogeneous decompositions that redistribute spatial evidence across scale/frequency coefficients.<n>We introduce a fundamentally different perspective: textbfhomogeneous image decomposition via Retinex theory, which factorizes an image into illumination and reflectance components within the emphsame spatial domain.
  arXiv  Detail & Related papers  (2026-05-14T22:19:52Z)
• Better with Less: Tackling Heterogeneous Multi-Modal Image Joint Pretraining via Conditioned and Degraded Masked Autoencoder [34.73963627819185]
  High-resolution (HR) joint optical and synthetic aperture radar (SAR) pretraining seeks modality synergy to mutually enhance single-source representations.<n>We propose CoDe-MAE, pioneering a textitbetter synergy with less alignment philosophy.<n>CoDe-MAE successfully prevents representation degradation and establishes new state-of-the-art performance across diverse single- and bi-modal downstream tasks.
  arXiv  Detail & Related papers  (2026-04-18T10:23:00Z)
• Universal Pansharpening Foundation Model [67.10467574892282]
  Pansharpening generates the high-resolution multi-spectral (MS) image by integrating spatial details from a texture-rich panchromatic (PAN) image and spectral attributes from a low-resolution MS image.<n>We present FoundPS, a universal pansharpening foundation model for satellite-agnostic and scene-robust fusion.
  arXiv  Detail & Related papers  (2026-03-04T08:30:15Z)
• TRACER: Texture-Robust Affordance Chain-of-Thought for Deformable-Object Refinement [37.77903164878976]
  TRACER is a Texture-Robust Affordance Chain-of-thought with dEformable-object Refinement framework.<n>It provides a cross-hierarchical mapping from semantic reasoning to appearance-robust and physically consistent functional region refinement.<n>Experiments conducted on the Fine-AGDDO15 dataset and a real-world robotic platform demonstrate that TRACER significantly improves affordance grounding precision.
  arXiv  Detail & Related papers  (2026-01-28T03:12:18Z)
• Joint Geometry-Appearance Human Reconstruction in a Unified Latent Space via Bridge Diffusion [57.09673862519791]
  This paper introduces textbfJGA-LBD, a novel framework that unifies the modeling of geometry and appearance into a joint latent representation.<n> Experiments demonstrate that JGA-LBD outperforms current state-of-the-art approaches in terms of both geometry fidelity and appearance quality.
  arXiv  Detail & Related papers  (2026-01-01T12:48:56Z)
• The Prism Hypothesis: Harmonizing Semantic and Pixel Representations via Unified Autoencoding [82.53463660564933]
  semantic encoders primarily capture low-frequency components that encode abstract meaning, whereas pixel encoders retain high-frequency information that conveys fine-grained detail.<n>We propose Unified Autoencoding (UAE), a model that harmonizes semantic structure and pixel details via an innovative frequency-band modulator.
  arXiv  Detail & Related papers  (2025-12-22T18:59:57Z)
• Manifold-aware Representation Learning for Degradation-agnostic Image Restoration [135.90908995927194]
  Image Restoration (IR) aims to recover high quality images from degraded inputs affected by various corruptions such as noise, blur, haze, rain, and low light conditions.<n>We present MIRAGE, a unified framework for all in one IR that explicitly decomposes the input feature space into three semantically aligned parallel branches.<n>This modular decomposition significantly improves generalization and efficiency across diverse degradations.
  arXiv  Detail & Related papers  (2025-05-24T12:52:10Z)
• AuxDet: Auxiliary Metadata Matters for Omni-Domain Infrared Small Target Detection [49.81255045696323]
  We present the Auxiliary Metadata Driven Infrared Small Target Detector (AuxDet)<n>AuxDet integrates metadata semantics with visual features, guiding adaptive representation learning for each sample.<n>Experiments on the challenging WideIRSTD-Full benchmark demonstrate that AuxDet consistently outperforms state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-05-21T07:02:05Z)
• ZoomNeXt: A Unified Collaborative Pyramid Network for Camouflaged Object Detection [70.11264880907652]
  Recent object (COD) attempts to segment objects visually blended into their surroundings, which is extremely complex and difficult in real-world scenarios. We propose an effective unified collaborative pyramid network that mimics human behavior when observing vague images and camouflaged zooming in and out. Our framework consistently outperforms existing state-of-the-art methods in image and video COD benchmarks.
  arXiv  Detail & Related papers  (2023-10-31T06:11:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.