Causal-Tune: Mining Causal Factors from Vision Foundation Models for Domain Generalized Semantic Segmentation

• URL: http://arxiv.org/abs/2512.16567v1
• Date: Thu, 18 Dec 2025 14:10:46 GMT
• Title: Causal-Tune: Mining Causal Factors from Vision Foundation Models for Domain Generalized Semantic Segmentation
• Authors: Yin Zhang, Yongqiang Zhang, Yaoyue Zheng, Bogdan Raducanu, Dan Liu,
• Abstract summary: Fine-tuning Vision Foundation Models (VFMs) with a small number of parameters has shown remarkable performance in Domain Generalized Semantic (DGSS)<n>We propose Causal-Tune, a novel fine-tuning strategy designed to extract causal factors and suppress non-causal ones from the features of VFMs.<n>Our method achieves superior performance under adverse weather conditions, improving +4.8% mIoU over the baseline in snow conditions.
• Score: 11.151384130523942
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Fine-tuning Vision Foundation Models (VFMs) with a small number of parameters has shown remarkable performance in Domain Generalized Semantic Segmentation (DGSS). Most existing works either train lightweight adapters or refine intermediate features to achieve better generalization on unseen domains. However, they both overlook the fact that long-term pre-trained VFMs often exhibit artifacts, which hinder the utilization of valuable representations and ultimately degrade DGSS performance. Inspired by causal mechanisms, we observe that these artifacts are associated with non-causal factors, which usually reside in the low- and high-frequency components of the VFM spectrum. In this paper, we explicitly examine the causal and non-causal factors of features within VFMs for DGSS, and propose a simple yet effective method to identify and disentangle them, enabling more robust domain generalization. Specifically, we propose Causal-Tune, a novel fine-tuning strategy designed to extract causal factors and suppress non-causal ones from the features of VFMs. First, we extract the frequency spectrum of features from each layer using the Discrete Cosine Transform (DCT). A Gaussian band-pass filter is then applied to separate the spectrum into causal and non-causal components. To further refine the causal components, we introduce a set of causal-aware learnable tokens that operate in the frequency domain, while the non-causal components are discarded. Finally, refined features are transformed back into the spatial domain via inverse DCT and passed to the next layer. Extensive experiments conducted on various cross-domain tasks demonstrate the effectiveness of Causal-Tune. In particular, our method achieves superior performance under adverse weather conditions, improving +4.8% mIoU over the baseline in snow conditions.

Related papers

• Improving Anomaly Detection with Foundation-Model Synthesis and Wavelet-Domain Attention [54.73850941855912]
  anomaly synthesis pipeline (FMAS) generates highly realistic anomalous samples without fine-tuning or class-specific training.<n>Wavelet Domain Attention Module (WDAM) exploits adaptive sub-band processing to enhance anomaly feature extraction.
  arXiv  Detail & Related papers  (2026-03-03T13:17:08Z)
• Rethinking Graph Domain Adaptation: A Spectral Contrastive Perspective [30.75684181076034]
  Graph neural networks (GNNs) have achieved remarkable success in various domains, yet they often struggle with domain adaptation.<n>We propose FracNet to decompose the original graph into high-frequency and low-frequency components and perform frequency-aware domain adaption.
  arXiv  Detail & Related papers  (2025-10-15T08:04:18Z)
• Wavelet-Guided Dual-Frequency Encoding for Remote Sensing Change Detection [67.84730634802204]
  Change detection in remote sensing imagery plays a vital role in various engineering applications, such as natural disaster monitoring, urban expansion tracking, and infrastructure management.<n>Most existing methods still rely on spatial-domain modeling, where the limited diversity of feature representations hinders the detection of subtle change regions.<n>We observe that frequency-domain feature modeling particularly in the wavelet domain amplify fine-grained differences in frequency components, enhancing the perception of edge changes that are challenging to capture in the spatial domain.
  arXiv  Detail & Related papers  (2025-08-07T11:14:16Z)
• Frequency Domain-Based Diffusion Model for Unpaired Image Dehazing [92.61216319417208]
  We propose a novel frequency domain-based diffusion model, named ours, for fully exploiting the beneficial knowledge in unpaired clear data.<n>Inspired by the strong generative ability shown by Diffusion Models (DMs), we tackle the dehazing task from the perspective of frequency domain reconstruction.
  arXiv  Detail & Related papers  (2025-07-02T01:22:46Z)
• Generalizable Multispectral Land Cover Classification via Frequency-Aware Mixture of Low-Rank Token Experts [22.75047167955269]
  We introduce Land-MoE, a novel approach for multispectral land cover classification (MLCC)<n>Land-MoE comprises two key modules: the mixture of low-rank token experts (MoLTE) and frequency-aware filters (FAF)
  arXiv  Detail & Related papers  (2025-05-20T08:52:28Z)
• FreSca: Scaling in Frequency Space Enhances Diffusion Models [55.75504192166779]
  This paper explores frequency-based control within latent diffusion models.<n>We introduce FreSca, a novel framework that decomposes noise difference into low- and high-frequency components.<n>FreSca operates without any model retraining or architectural change, offering model- and task-agnostic control.
  arXiv  Detail & Related papers  (2025-04-02T22:03:11Z)
• Frequency Domain Enhanced U-Net for Low-Frequency Information-Rich Image Segmentation in Surgical and Deep-Sea Exploration Robots [34.28684917337352]
  We address the differences in frequency band sensitivity between CNNs and the human visual system.<n>We propose a wavelet adaptive spectrum fusion (WASF) method inspired by biological vision mechanisms to balance cross-frequency image features.<n>We develop the FE-UNet model, which employs a SAM2 backbone network and incorporates fine-tuned Hiera-Large modules to ensure segmentation accuracy.
  arXiv  Detail & Related papers  (2025-02-06T07:24:34Z)
• Integrating Frequency Guidance into Multi-source Domain Generalization for Bearing Fault Diagnosis [24.85752780864944]
  We propose the Fourier-based Augmentation Reconstruction Network, namely FARNet.<n>The network comprises an amplitude spectrum sub-network and a phase spectrum sub-network, sequentially reducing the discrepancy between the source and target domains.<n>To refine the decision boundary of our model output compared to conventional triplet loss, we propose a manifold triplet loss to contribute to generalization.
  arXiv  Detail & Related papers  (2025-02-01T20:23:03Z)
• Multi-view Adversarial Discriminator: Mine the Non-causal Factors for Object Detection in Unseen Domains [36.4342793435982]
  We present an idea to remove non-causal factors from common features by multi-view adversarial training on source domains. We propose a Multi-view Adversarial Discriminator (MAD) based domain generalization model. Our MAD obtains state-of-the-art performance on six benchmarks.
  arXiv  Detail & Related papers  (2023-04-06T09:20:28Z)
• FAMLP: A Frequency-Aware MLP-Like Architecture For Domain Generalization [73.41395947275473]
  We propose a novel frequency-aware architecture, in which the domain-specific features are filtered out in the transformed frequency domain. Experiments on three benchmarks demonstrate significant performance, outperforming the state-of-the-art methods by a margin of 3%, 4% and 9%, respectively.
  arXiv  Detail & Related papers  (2022-03-24T07:26:29Z)
• Deep Frequency Filtering for Domain Generalization [55.66498461438285]
  Deep Neural Networks (DNNs) have preferences for some frequency components in the learning process. We propose Deep Frequency Filtering (DFF) for learning domain-generalizable features. We show that applying our proposed DFF on a plain baseline outperforms the state-of-the-art methods on different domain generalization tasks.
  arXiv  Detail & Related papers  (2022-03-23T05:19:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.