Local Precise Refinement: A Dual-Gated Mixture-of-Experts for Enhancing Foundation Model Generalization against Spectral Shifts

• URL: http://arxiv.org/abs/2603.13352v1
• Date: Sun, 08 Mar 2026 04:53:05 GMT
• Title: Local Precise Refinement: A Dual-Gated Mixture-of-Experts for Enhancing Foundation Model Generalization against Spectral Shifts
• Abstract summary: Domain Generalization Semantic (DGSS) in spectral remote sensing is severely challenged by spectral shifts across diverse acquisition conditions.<n>We propose SpectralMoE, a novel PEFT framework for DGSS that performs fine-grained, spatially-adaptive refinement of a foundation model's features.
• Score: 26.40090723572808
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Domain Generalization Semantic Segmentation (DGSS) in spectral remote sensing is severely challenged by spectral shifts across diverse acquisition conditions, which cause significant performance degradation for models deployed in unseen domains. While Parameter-Efficient Fine-Tuning (PEFT) on foundation models is a promising direction, existing methods employ global, homogeneous adjustments. This "one-size-fits-all" tuning struggles with the spatial heterogeneity of land cover, causing semantic confusion. We argue that the key to robust DGSS lies not in a single global adaptation, but in performing fine-grained, spatially-adaptive refinement of a foundation model's features. To achieve this, we propose SpectralMoE, a novel PEFT framework for DGSS. It operationalizes this principle by utilizing a Mixture-of-Experts (MoE) architecture to perform local precise refinement on the foundation model's features, incorporating depth features estimated from selected RGB bands of the spectral remote sensing imagery to guide the fine-tuning process. Specifically, SpectralMoE employs a dual-gated MoE architecture that independently routes visual and depth features to top-k selected experts for specialized refinement, enabling modality-specific adjustments. A subsequent cross-attention mechanism then judiciously fuses the refined structural cues into the visual stream, mitigating semantic ambiguities caused by spectral variations. Extensive experiments show that SpectralMoE sets a new state-of-the-art on multiple DGSS benchmarks across hyperspectral, multispectral, and RGB remote sensing imagery.

Related papers

• MSP-Net: Manifold-Guided Spectral Prompt Network for Hyperspectral Object Tracking [11.03462046191901]
  Hyperspectral object tracking offers unique advantages for target discrimination in complex scenes.<n>Existing methods treat hyperspectral images as multi-channel extensions of RGB images, performing feature fusion in fixed band order.<n>We propose the Manifold-Guided Spectral Prompt Network (MSP-Net)<n> Experiments on HOT2020 and HOT2023 demonstrate MSP-Net achieves AUC and Precision exceeding 0.80 and 0.96, respectively.
  arXiv  Detail & Related papers  (2026-08-10T13:09:25Z)
• Spectral Property-Driven Data Augmentation for Hyperspectral Single-Source Domain Generalization [13.335941336922446]
  Hyperspectral images benefit from numerous spectral channels that provide rich information for classification.<n>The increased dimensionality and sensor variability make them more sensitive to distributional discrepancies across domains.<n>We propose a spectral property-driven data augmentation (SPDDA) that explicitly accounts for the inherent properties of HSI.
  arXiv  Detail & Related papers  (2026-03-17T15:30:28Z)
• SR$^{2}$-Net: A General Plug-and-Play Model for Spectral Refinement in Hyperspectral Image Super-Resolution [3.4888894498274747]
  HSI-SR aims to enhance spatial resolution while preserving spectrally faithful and physically plausible characteristics.<n>These methods often neglect spectral consistency across bands, leading to spurious oscillations and physically implausible artifacts.<n>We propose a lightweight plug-and-play, physically priors Spectral Rectification Super-Resolution Network (SR$2$-Net) to address this issue.
  arXiv  Detail & Related papers  (2026-01-29T07:00:00Z)
• Any-Optical-Model: A Universal Foundation Model for Optical Remote Sensing [24.03278912134978]
  We propose Any Optical Model (AOM) to accommodate arbitrary band compositions, sensor types, and resolution scales.<n>AOM consistently achieves state-of-the-art (SOTA) performance under challenging conditions such as band missing, cross sensor, and cross resolution settings.
  arXiv  Detail & Related papers  (2025-12-19T04:21:01Z)
• Hybrid Deep Learning for Hyperspectral Single Image Super-Resolution [9.467214671383875]
  We introduce Spectral-Spatial Unmixing Fusion (SSUF), a novel module that can be seamlessly integrated into standard 2D convolutional architectures.<n>The SSUF combines spectral unmixing with spectral--spatial feature extraction and guides a ResNet-based convolutional neural network for improved reconstruction.<n> Experiments on three public remote sensing hyperspectral datasets demonstrate that the proposed hybrid deep learning model achieves competitive performance.
  arXiv  Detail & Related papers  (2025-09-26T08:28:07Z)
• SpectralX: Parameter-efficient Domain Generalization for Spectral Remote Sensing Foundation Models [27.717648142854696]
  SpectralX is an innovative parameter-efficient fine-tuning framework that adapt existing RSFMs as backbone.<n>We develop an Attribute-oriented Mixture of Adapter (AoMoA) that dynamically aggregates multi-attribute expert knowledge.<n>By iteratively querying low-level semantic features with high-level representations, the model learns to focus on task-beneficial attributes.
  arXiv  Detail & Related papers  (2025-08-03T12:14:38Z)
• SpectrumFM: Redefining Spectrum Cognition via Foundation Modeling [65.65474629224558]
  We propose a spectrum foundation model, termed SpectrumFM, which provides a new paradigm for spectrum cognition.<n>An innovative spectrum encoder that exploits the convolutional neural networks is proposed to effectively capture both fine-grained local signal structures and high-level global dependencies in the spectrum data.<n>Two novel self-supervised learning tasks, namely masked reconstruction and next-slot signal prediction, are developed for pre-training SpectrumFM, enabling the model to learn rich and transferable representations.
  arXiv  Detail & Related papers  (2025-08-02T14:40:50Z)
• Generalizable Multispectral Land Cover Classification via Frequency-Aware Mixture of Low-Rank Token Experts [22.75047167955269]
  We introduce Land-MoE, a novel approach for multispectral land cover classification (MLCC)<n>Land-MoE comprises two key modules: the mixture of low-rank token experts (MoLTE) and frequency-aware filters (FAF)
  arXiv  Detail & Related papers  (2025-05-20T08:52:28Z)
• SpectrumFM: A Foundation Model for Intelligent Spectrum Management [99.08036558911242]
  Existing intelligent spectrum management methods, typically based on small-scale models, suffer from notable limitations in recognition accuracy, convergence speed, and generalization.<n>This paper proposes a novel spectrum foundation model, termed SpectrumFM, establishing a new paradigm for spectrum management.<n>Experiments demonstrate that SpectrumFM achieves superior performance in terms of accuracy, robustness, adaptability, few-shot learning efficiency, and convergence speed.
  arXiv  Detail & Related papers  (2025-05-02T04:06:39Z)
• CARL: Camera-Agnostic Representation Learning for Spectral Image Analysis [69.02751635551724]
  Spectral imaging offers promising applications across diverse domains, including medicine and urban scene understanding.<n> variability in channel dimensionality and captured wavelengths among spectral cameras impede the development of AI-driven methodologies.<n>We introduce CARL, a model for Camera-Agnostic Representation Learning across RGB, multispectral, and hyperspectral imaging modalities.
  arXiv  Detail & Related papers  (2025-04-27T13:06:40Z)
• FreSca: Scaling in Frequency Space Enhances Diffusion Models [55.75504192166779]
  This paper explores frequency-based control within latent diffusion models.<n>We introduce FreSca, a novel framework that decomposes noise difference into low- and high-frequency components.<n>FreSca operates without any model retraining or architectural change, offering model- and task-agnostic control.
  arXiv  Detail & Related papers  (2025-04-02T22:03:11Z)
• Unleashing Correlation and Continuity for Hyperspectral Reconstruction from RGB Images [64.80875911446937]
  We propose a Correlation and Continuity Network (CCNet) for HSI reconstruction from RGB images.<n>For the correlation of local spectrum, we introduce the Group-wise Spectral Correlation Modeling (GrSCM) module.<n>For the continuity of global spectrum, we design the Neighborhood-wise Spectral Continuity Modeling (NeSCM) module.
  arXiv  Detail & Related papers  (2025-01-02T15:14:40Z)
• SpectralGPT: Spectral Remote Sensing Foundation Model [60.023956954916414]
  A universal RS foundation model, named SpectralGPT, is purpose-built to handle spectral RS images using a novel 3D generative pretrained transformer (GPT) Compared to existing foundation models, SpectralGPT accommodates input images with varying sizes, resolutions, time series, and regions in a progressive training fashion, enabling full utilization of extensive RS big data. Our evaluation highlights significant performance improvements with pretrained SpectralGPT models, signifying substantial potential in advancing spectral RS big data applications within the field of geoscience.
  arXiv  Detail & Related papers  (2023-11-13T07:09:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.