MOSAIC: Masked Objective with Selective Adaptation for In-domain Contrastive Learning

• URL: http://arxiv.org/abs/2510.16797v1
• Date: Sun, 19 Oct 2025 11:24:03 GMT
• Title: MOSAIC: Masked Objective with Selective Adaptation for In-domain Contrastive Learning
• Authors: Vera Pavlova, Mohammed Makhlouf,
• Abstract summary: MOSAIC is a multi-stage framework for domain adaptation of sentence embedding models.<n>By jointly optimizing masked language modeling (MLM) and contrastive objectives within a unified training pipeline, our method enables effective learning of domain-relevant representations.
• Score: 1.2891210250935148
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce MOSAIC (Masked Objective with Selective Adaptation for In-domain Contrastive learning), a multi-stage framework for domain adaptation of sentence embedding models that incorporates joint domain-specific masked supervision. Our approach addresses the challenges of adapting large-scale general-domain sentence embedding models to specialized domains. By jointly optimizing masked language modeling (MLM) and contrastive objectives within a unified training pipeline, our method enables effective learning of domain-relevant representations while preserving the robust semantic discrimination properties of the original model. We empirically validate our approach on both high-resource and low-resource domains, achieving improvements up to 13.4% in NDCG@10 (Normalized Discounted Cumulative Gain) over strong general-domain baselines. Comprehensive ablation studies further demonstrate the effectiveness of each component, highlighting the importance of balanced joint supervision and staged adaptation.

Related papers

• OMUDA: Omni-level Masking for Unsupervised Domain Adaptation in Semantic Segmentation [29.520497497017015]
  We propose Omni-level Masking for Unsupervised Domain Adaptation (OMUDA), a unified framework that introduces hierarchical masking strategies across distinct representation levels.<n>OMUDA comprises: 1) a Context-Aware Masking (CAM) strategy that adaptively distinguishes foreground from background to balance global context and local details; 2) a Feature Distillation Masking (FDM) strategy that enhances robust and consistent feature learning through knowledge transfer from pre-trained models; and 3) a Class Decoupling Masking (CDM) strategy that mitigates the impact of noisy pseudo-labels by explicitly modeling class-wise uncertainty.
  arXiv  Detail & Related papers  (2025-12-13T12:01:23Z)
• SoMA: Singular Value Decomposed Minor Components Adaptation for Domain Generalizable Representation Learning [6.262268096839562]
  Domain generalization aims to adapt a model using one or multiple source domains to ensure robust performance in unseen target domains.<n>Existing PEFT methods struggle to strike a balance between preserving generalizable components of the pre-trained model and learning task-specific features.<n>We introduce Singular Value De Minor Components Adaptation (SoMA), an approach that selectively tunes minor singular components while keeping the residual parts frozen.
  arXiv  Detail & Related papers  (2024-12-05T11:17:57Z)
• Mitigate Domain Shift by Primary-Auxiliary Objectives Association for Generalizing Person ReID [39.98444065846305]
  ReID models struggle in learning domain-invariant representation solely through training on an instance classification objective. We introduce a method that guides model learning of the primary ReID instance classification objective by a concurrent auxiliary learning objective on weakly labeled pedestrian saliency detection. Our model can be extended with the recent test-time diagram to form the PAOA+, which performs on-the-fly optimization against the auxiliary objective.
  arXiv  Detail & Related papers  (2023-10-24T15:15:57Z)
• Open-Set Domain Adaptation with Visual-Language Foundation Models [51.49854335102149]
  Unsupervised domain adaptation (UDA) has proven to be very effective in transferring knowledge from a source domain to a target domain with unlabeled data. Open-set domain adaptation (ODA) has emerged as a potential solution to identify these classes during the training phase.
  arXiv  Detail & Related papers  (2023-07-30T11:38:46Z)
• IDA: Informed Domain Adaptive Semantic Segmentation [51.12107564372869]
  We propose an Domain Informed Adaptation (IDA) model, a self-training framework that mixes the data based on class-level segmentation performance. In our IDA model, the class-level performance is tracked by an expected confidence score (ECS) and we then use a dynamic schedule to determine the mixing ratio for data in different domains. Our proposed method is able to outperform the state-of-the-art UDA-SS method by a margin of 1.1 mIoU in the adaptation of GTA-V to Cityscapes and of 0.9 mIoU in the adaptation of SYNTHIA to City
  arXiv  Detail & Related papers  (2023-03-05T18:16:34Z)
• Multi-level Consistency Learning for Semi-supervised Domain Adaptation [85.90600060675632]
  Semi-supervised domain adaptation (SSDA) aims to apply knowledge learned from a fully labeled source domain to a scarcely labeled target domain. We propose a Multi-level Consistency Learning framework for SSDA.
  arXiv  Detail & Related papers  (2022-05-09T06:41:18Z)
• From Big to Small: Adaptive Learning to Partial-Set Domains [94.92635970450578]
  Domain adaptation targets at knowledge acquisition and dissemination from a labeled source domain to an unlabeled target domain under distribution shift. Recent advances show that deep pre-trained models of large scale endow rich knowledge to tackle diverse downstream tasks of small scale. This paper introduces Partial Domain Adaptation (PDA), a learning paradigm that relaxes the identical class space assumption to that the source class space subsumes the target class space.
  arXiv  Detail & Related papers  (2022-03-14T07:02:45Z)
• Cluster, Split, Fuse, and Update: Meta-Learning for Open Compound Domain Adaptive Semantic Segmentation [102.42638795864178]
  We propose a principled meta-learning based approach to OCDA for semantic segmentation. We cluster target domain into multiple sub-target domains by image styles, extracted in an unsupervised manner. A meta-learner is thereafter deployed to learn to fuse sub-target domain-specific predictions, conditioned upon the style code. We learn to online update the model by model-agnostic meta-learning (MAML) algorithm, thus to further improve generalization.
  arXiv  Detail & Related papers  (2020-12-15T13:21:54Z)
• Exploiting Diverse Characteristics and Adversarial Ambivalence for Domain Adaptive Segmentation [20.13548631627542]
  Adapting semantic segmentation models to new domains is an important but challenging problem. We propose a condition-guided adaptation framework that is empowered by a special progressive adversarial training mechanism and a novel self-training policy. We evaluate our method on various adaptation scenarios where the target images vary in weather conditions.
  arXiv  Detail & Related papers  (2020-12-10T11:50:59Z)
• Adaptively-Accumulated Knowledge Transfer for Partial Domain Adaptation [66.74638960925854]
  Partial domain adaptation (PDA) deals with a realistic and challenging problem when the source domain label space substitutes the target domain. We propose an Adaptively-Accumulated Knowledge Transfer framework (A$2$KT) to align the relevant categories across two domains.
  arXiv  Detail & Related papers  (2020-08-27T00:53:43Z)
• Improve Unsupervised Domain Adaptation with Mixup Training [18.329571222689562]
  We study the problem of utilizing a relevant source domain with abundant labels to build predictive modeling for an unannotated target domain. Recent work observe that the popular adversarial approach of learning domain-invariant features is insufficient to achieve desirable target domain performance. We propose to enforce training constraints across domains using mixup formulation to directly address the generalization performance for target data.
  arXiv  Detail & Related papers  (2020-01-03T01:21:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.