Revisiting Cross-Attention Mechanisms: Leveraging Beneficial Noise for Domain-Adaptive Learning

• URL: http://arxiv.org/abs/2603.17474v1
• Date: Wed, 18 Mar 2026 08:28:14 GMT
• Title: Revisiting Cross-Attention Mechanisms: Leveraging Beneficial Noise for Domain-Adaptive Learning
• Abstract summary: Unsupervised Domain Adaptation (UDA) seeks to transfer knowledge from a labeled source domain to an unlabeled target domain.<n>Existing cross-attention-based transformers can align features across domains, yet they struggle to preserve content semantics under large appearance and scale variations.<n>We introduce the concept of beneficial noise, which regularizes cross-attention by injecting controlled perturbations.
• Score: 19.18410733239502
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Unsupervised Domain Adaptation (UDA) seeks to transfer knowledge from a labeled source domain to an unlabeled target domain but often suffers from severe domain and scale gaps that degrade performance. Existing cross-attention-based transformers can align features across domains, yet they struggle to preserve content semantics under large appearance and scale variations. To explicitly address these challenges, we introduce the concept of beneficial noise, which regularizes cross-attention by injecting controlled perturbations, encouraging the model to ignore style distractions and focus on content. We propose the Domain-Adaptive Cross-Scale Matching (DACSM) framework, which consists of a Domain-Adaptive Transformer (DAT) for disentangling domain-shared content from domain-specific style, and a Cross-Scale Matching (CSM) module that adaptively aligns features across multiple resolutions. DAT incorporates beneficial noise into cross-attention, enabling progressive domain translation with enhanced robustness, yielding content-consistent and style-invariant representations. Meanwhile, CSM ensures semantic consistency under scale changes. Extensive experiments on VisDA-2017, Office-Home, and DomainNet demonstrate that DACSM achieves state-of-the-art performance, with up to +2.3% improvement over CDTrans on VisDA-2017. Notably, DACSM achieves a +5.9% gain on the challenging "truck" class of VisDA, evidencing the strength of beneficial noise in handling scale discrepancies. These results highlight the effectiveness of combining domain translation, beneficial-noise-enhanced attention, and scale-aware alignment for robust cross-domain representation learning.

Related papers

• PCaM: A Progressive Focus Attention-Based Information Fusion Method for Improving Vision Transformer Domain Adaptation [18.973817257766793]
  Unsupervised Domain Adaptation (UDA) aims to transfer knowledge from a labeled source domain to an unlabeled target domain.<n>We propose the Progressive Focus Cross-Attention Mechanism (PCaM)<n>PCaM progressively filters out background information during cross-attention, allowing the model to focus on and fuse discriminative foreground semantics across domains.
  arXiv  Detail & Related papers  (2025-05-27T09:48:29Z)
• Cross-Domain Diffusion with Progressive Alignment for Efficient Adaptive Retrieval [52.67656818203429]
  Unsupervised efficient domain adaptive retrieval aims to transfer knowledge from a labeled source domain to an unlabeled target domain.<n>Existing methods fail to address potential noise in the target domain, and directly align high-level features across domains.<n>We propose a novel Cross-Domain Diffusion with Progressive Alignment method (COUPLE) to address these challenges.
  arXiv  Detail & Related papers  (2025-05-20T04:17:39Z)
• DA-Mamba: Domain Adaptive Hybrid Mamba-Transformer Based One-Stage Object Detection [0.3683202928838613]
  We present the first domain-adaptive Mamba-based one-stage object detection model, DA-Mamba.<n>Inspired by the global modeling and linear complexity of the Mamba architecture, we present the first domain-adaptive Mamba-based one-stage object detection model, DA-Mamba.
  arXiv  Detail & Related papers  (2025-02-16T15:58:54Z)
• Exploring Semantic Consistency and Style Diversity for Domain Generalized Semantic Segmentation [4.850207292777464]
  Domain Generalized Semantic aims to enhance the generalization of semantic segmentation across unknown target domains.<n>We introduce SCSD for Semantic Consistency prediction and Style Diversity generalization.<n>SCSD significantly outperforms existing state-of-theart methods.
  arXiv  Detail & Related papers  (2024-12-16T18:20:06Z)
• Language-Guided Instance-Aware Domain-Adaptive Panoptic Segmentation [44.501770535446624]
  Key challenge in panoptic domain adaptation is reducing the domain gap between a labeled source and an unlabeled target domain.<n>We focus on incorporating instance-level adaptation via a novel cross-domain mixing strategy IMix.<n>We present an end-to-end model incorporating these two mechanisms called LIDAPS, achieving state-of-the-art results on all popular panoptic UDA benchmarks.
  arXiv  Detail & Related papers  (2024-04-04T20:42:49Z)
• Fine-Grained Unsupervised Cross-Modality Domain Adaptation for Vestibular Schwannoma Segmentation [3.0081059328558624]
  We focus on introducing a fine-grained unsupervised framework for domain adaptation. We propose to use a vector to control the generator to synthesize a fake image with given features. And then, we can apply various augmentations to the dataset by searching the feature dictionary.
  arXiv  Detail & Related papers  (2023-11-25T18:08:59Z)
• Exploring Consistency in Cross-Domain Transformer for Domain Adaptive Semantic Segmentation [51.10389829070684]
  Domain gap can cause discrepancies in self-attention. Due to this gap, the transformer attends to spurious regions or pixels, which deteriorates accuracy on the target domain. We propose adaptation on attention maps with cross-domain attention layers.
  arXiv  Detail & Related papers  (2022-11-27T02:40:33Z)
• Unsupervised Domain Adaptation via Style-Aware Self-intermediate Domain [52.783709712318405]
  Unsupervised domain adaptation (UDA) has attracted considerable attention, which transfers knowledge from a label-rich source domain to a related but unlabeled target domain.<n>We propose a novel style-aware feature fusion method (SAFF) to bridge the large domain gap and transfer knowledge while alleviating the loss of class-discnative information.
  arXiv  Detail & Related papers  (2022-09-05T10:06:03Z)
• Safe Self-Refinement for Transformer-based Domain Adaptation [73.8480218879]
  Unsupervised Domain Adaptation (UDA) aims to leverage a label-rich source domain to solve tasks on a related unlabeled target domain. It is a challenging problem especially when a large domain gap lies between the source and target domains. We propose a novel solution named SSRT (Safe Self-Refinement for Transformer-based domain adaptation), which brings improvement from two aspects.
  arXiv  Detail & Related papers  (2022-04-16T00:15:46Z)
• Amplitude Spectrum Transformation for Open Compound Domain Adaptive Semantic Segmentation [62.68759523116924]
  Open compound domain adaptation (OCDA) has emerged as a practical adaptation setting. We propose a novel feature space Amplitude Spectrum Transformation (AST)
  arXiv  Detail & Related papers  (2022-02-09T05:40:34Z)
• AFAN: Augmented Feature Alignment Network for Cross-Domain Object Detection [90.18752912204778]
  Unsupervised domain adaptation for object detection is a challenging problem with many real-world applications. We propose a novel augmented feature alignment network (AFAN) which integrates intermediate domain image generation and domain-adversarial training. Our approach significantly outperforms the state-of-the-art methods on standard benchmarks for both similar and dissimilar domain adaptations.
  arXiv  Detail & Related papers  (2021-06-10T05:01:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.