Lean Learning Beyond Clouds: Efficient Discrepancy-Conditioned Optical-SAR Fusion for Semantic Segmentation

• URL: http://arxiv.org/abs/2603.20811v1
• Date: Sat, 21 Mar 2026 13:14:40 GMT
• Title: Lean Learning Beyond Clouds: Efficient Discrepancy-Conditioned Optical-SAR Fusion for Semantic Segmentation
• Abstract summary: We propose EDC, an efficiency-oriented and discrepancy-conditioned optical-SAR semantic segmentation framework.<n> EDC achieves superior accuracy and efficiency, improving mIoU by 0.56% and 0.88% on M3M-CR and WHU-OPT-SAR, respectively.
• Score: 26.501346227659866
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cloud occlusion severely degrades the semantic integrity of optical remote sensing imagery. While incorporating Synthetic Aperture Radar (SAR) provides complementary observations, achieving efficient global modeling and reliable cross-modal fusion under cloud interference remains challenging. Existing methods rely on dense global attention to capture long-range dependencies, yet such aggregation indiscriminately propagates cloud-induced noise. Improving robustness typically entails enlarging model capacity, which further increases computational overhead. Given the large-scale and high-resolution nature of remote sensing applications, such computational demands hinder practical deployment, leading to an efficiency-reliability trade-off. To address this dilemma, we propose EDC, an efficiency-oriented and discrepancy-conditioned optical-SAR semantic segmentation framework. A tri-stream encoder with Carrier Tokens enables compact global context modeling with reduced complexity. To prevent noise contamination, we introduce a Discrepancy-Conditioned Hybrid Fusion (DCHF) mechanism that selectively suppresses unreliable regions during global aggregation. In addition, an auxiliary cloud removal branch with teacher-guided distillation enhances semantic consistency under occlusion. Extensive experiments demonstrate that EDC achieves superior accuracy and efficiency, improving mIoU by 0.56\% and 0.88\% on M3M-CR and WHU-OPT-SAR, respectively, while reducing the number of parameters by 46.7\% and accelerating inference by 1.98$\times$. Our implementation is available at https://github.com/mengcx0209/EDC.

Related papers

• IB-HFN: Information Bottleneck-Driven SAR-Optical Fusion Network for High-Fidelity Cloud Removal [18.935012033334477]
  We propose an Information Bottleneck-driven High-Fidelity Network (IB-HFN) for SAR-assisted optical cloud removal.<n>IB-HFN employs a dual-stream backbone to preserve modality-specific representations before deep semantic fusion.<n> Experiments on the SEN12MS-CR dataset under challenging-temporal splits demonstrate that IB-HFN achieves superior structural preservation and spectral fidelity over existing methods.
  arXiv  Detail & Related papers  (2026-06-08T11:19:06Z)
• Efficient One-Step Diffusion Restoration Model with Compact Token Compression and Linear Attention [66.63806505114263]
  Existing Real-ISR methods inherit the dense latent representations and quadratic-cost global modeling paradigm.<n>We argue that the key bottleneck lies in excessive token redundancy and costly token interactions during high-resolution restoration.<n>We introduce a linear-attention DiT with LoRA fine-tuning, enabling efficient high-resolution restoration with linear-complexity token mixing.
  arXiv  Detail & Related papers  (2026-05-22T10:07:02Z)
• OmniISR: A Unified Framework for Centralized and Federated Learning via Intermediate Supervision and Regularization [58.03221830946145]
  We propose OmniISR, a unified framework that fuses pure CL, pure FL, and hybrid CL-FL training modes.<n>We show that OmniISR consistently improves model performance in both centralized and federated paradigms.
  arXiv  Detail & Related papers  (2026-05-19T04:13:27Z)
• SlimDiffSR: Toward Lightweight and Efficient Remote Sensing Image Super-Resolution via Diffusion Model Distillation [10.518150741228862]
  SlimDiffSR is a lightweight and efficient diffusion-based framework for real-world remote sensing image super-resolution.<n>It attains up to $200times$ inference acceleration and a $20times$ reduction in model parameters compared with multi-step diffusion models.
  arXiv  Detail & Related papers  (2026-05-04T03:54:54Z)
• Better with Less: Tackling Heterogeneous Multi-Modal Image Joint Pretraining via Conditioned and Degraded Masked Autoencoder [34.73963627819185]
  High-resolution (HR) joint optical and synthetic aperture radar (SAR) pretraining seeks modality synergy to mutually enhance single-source representations.<n>We propose CoDe-MAE, pioneering a textitbetter synergy with less alignment philosophy.<n>CoDe-MAE successfully prevents representation degradation and establishes new state-of-the-art performance across diverse single- and bi-modal downstream tasks.
  arXiv  Detail & Related papers  (2026-04-18T10:23:00Z)
• SADER: Structure-Aware Diffusion Framework with DEterministic Resampling for Multi-Temporal Remote Sensing Cloud Removal [24.84158171817714]
  Diffusion-based models have emerged as a dominant paradigm for remote sensing cloud removal.<n>We propose SADER, a structure-aware diffusion framework for multi-temporal remote sensing cloud removal.<n>Experiments on multiple multi-temporal datasets demonstrate that SADER consistently outperforms state-of-the-art cloud removal methods.
  arXiv  Detail & Related papers  (2026-01-31T06:06:18Z)
• SKANet: A Cognitive Dual-Stream Framework with Adaptive Modality Fusion for Robust Compound GNSS Interference Classification [47.20483076887704]
  Global Navigation Satellite Systems (GNSS) face growing threats from sophisticated jamming interference.<n>We propose a cognitive deep learning framework built upon a dual-stream architecture that integrates Time-Frequency Images (TFIs) and Power Spectral Density (PSD)<n>We show that SKANet achieves an overall accuracy of 96.99%, exhibiting superior robustness for compound jamming classification.
  arXiv  Detail & Related papers  (2026-01-19T07:42:45Z)
• Task-Driven Prompt Learning: A Joint Framework for Multi-modal Cloud Removal and Segmentation [11.468907022707013]
  TDP-CR is a task-driven framework that jointly performs cloud removal and land-cover segmentation.<n>Central to our approach is a Prompt-Guided Fusion mechanism, which utilizes a learnable degradation prompt to encode cloud thickness and spatial uncertainty.<n>Experiments on the LuojiaSET-OSFCR dataset demonstrate the superiority of our framework.
  arXiv  Detail & Related papers  (2026-01-17T13:32:38Z)
• SDAR: A Synergistic Diffusion-AutoRegression Paradigm for Scalable Sequence Generation [62.14510717860079]
  We propose a Synergistic Diffusion-Autoregression paradigm that unifies the training efficiency of autoregressive models with the parallel inference capability of diffusion.<n>SDAR performs a lightweight paradigm conversion that transforms a well-trained autoregressive (AR) model into a blockwise diffusion model through brief, data-efficient adaptation.<n>Building on this insight, SDAR achieves efficient AR-to-diffusion conversion with minimal cost, preserving AR-level performance while enabling parallel generation.
  arXiv  Detail & Related papers  (2025-10-07T17:29:28Z)
• Pushing Trade-Off Boundaries: Compact yet Effective Remote Sensing Change Detection [5.686242146196292]
  We propose FlickCD, which means quick flick then get great results, pushing the boundaries of the performance-resource trade-off.<n>FlickCD reduces computational and storage overheads by more than an order of magnitude while achieving state-of-the-art (SOTA) performance or incurring only a minor (1% F1) accuracy trade-off.
  arXiv  Detail & Related papers  (2025-06-26T09:06:52Z)
• FUSE: Label-Free Image-Event Joint Monocular Depth Estimation via Frequency-Decoupled Alignment and Degradation-Robust Fusion [92.4205087439928]
  Image-event joint depth estimation methods leverage complementary modalities for robust perception, yet face challenges in generalizability.<n>We propose the Self-supervised Transfer (PST) and the FrequencyDe-coupled Fusion module (FreDF)<n>PST establishes cross-modal knowledge transfer through latent space alignment with image foundation models, effectively mitigating data scarcity.<n>FreDF explicitly decouples high-frequency edge features from low-frequency structural components, resolving modality-specific frequency mismatches.<n>This combined approach enables FUSE to construct a universal image-event that only requires lightweight decoder adaptation for target datasets.
  arXiv  Detail & Related papers  (2025-03-25T15:04:53Z)
• Theoretical Insights in Model Inversion Robustness and Conditional Entropy Maximization for Collaborative Inference Systems [89.35169042718739]
  collaborative inference enables end users to leverage powerful deep learning models without exposure of sensitive raw data to cloud servers.<n>Recent studies have revealed that these intermediate features may not sufficiently preserve privacy, as information can be leaked and raw data can be reconstructed via model inversion attacks (MIAs)<n>This work first theoretically proves that the conditional entropy of inputs given intermediate features provides a guaranteed lower bound on the reconstruction mean square error (MSE) under any MIA.<n>Then, we derive a differentiable and solvable measure for bounding this conditional entropy based on the Gaussian mixture estimation and propose a conditional entropy algorithm to enhance the inversion robustness
  arXiv  Detail & Related papers  (2025-03-01T07:15:21Z)
• Momentum Contrastive Autoencoder: Using Contrastive Learning for Latent Space Distribution Matching in WAE [51.09507030387935]
  Wasserstein autoencoder (WAE) shows that matching two distributions is equivalent to minimizing a simple autoencoder (AE) loss under the constraint that the latent space of this AE matches a pre-specified prior distribution. We propose to use the contrastive learning framework that has been shown to be effective for self-supervised representation learning, as a means to resolve this problem. We show that using the contrastive learning framework to optimize the WAE loss achieves faster convergence and more stable optimization compared with existing popular algorithms for WAE.
  arXiv  Detail & Related papers  (2021-10-19T22:55:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.