A Strategy for Label Alignment in Deep Neural Networks

• URL: http://arxiv.org/abs/2410.04722v2
• Date: Wed, 12 Mar 2025 15:04:03 GMT
• Title: A Strategy for Label Alignment in Deep Neural Networks
• Authors: Xuanrui Zeng,
• Abstract summary: In this work, we derive an alternative formulation of the original adaptation algorithm exploiting label alignment suitable for deep neural network.<n>We also perform experiments to demonstrate that our approach achieves comparable performance to mainstream unsupervised domain adaptation methods while having stabler convergence.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: One recent research demonstrated successful application of the label alignment property for unsupervised domain adaptation in a linear regression settings. Instead of regularizing representation learning to be domain invariant, the research proposed to regularize the linear regression model to align with the top singular vectors of the data matrix from the target domain. In this work we expand upon this idea and generalize it to the case of deep learning, where we derive an alternative formulation of the original adaptation algorithm exploiting label alignment suitable for deep neural network. We also perform experiments to demonstrate that our approach achieves comparable performance to mainstream unsupervised domain adaptation methods while having stabler convergence. All experiments and implementations in our work can be found at the following codebase: https://github.com/xuanrui-work/DeepLabelAlignment.

Related papers

• Recurrent Aligned Network for Generalized Pedestrian Trajectory Prediction [43.98749443981923]
  Pedestrian trajectory prediction is a crucial component in computer vision and robotics. Previous studies have tried to tackle this problem by leveraging a portion of the trajectory data from the target domain to adapt the model. We introduce a Recurrent Aligned Network(RAN) to minimize the domain gap through domain alignment.
  arXiv  Detail & Related papers  (2024-03-09T06:17:09Z)
• Bi-discriminator Domain Adversarial Neural Networks with Class-Level Gradient Alignment [87.8301166955305]
  We propose a novel bi-discriminator domain adversarial neural network with class-level gradient alignment. BACG resorts to gradient signals and second-order probability estimation for better alignment of domain distributions. In addition, inspired by contrastive learning, we develop a memory bank-based variant, i.e. Fast-BACG, which can greatly shorten the training process.
  arXiv  Detail & Related papers  (2023-10-21T09:53:17Z)
• Activate and Reject: Towards Safe Domain Generalization under Category Shift [71.95548187205736]
  We study a practical problem of Domain Generalization under Category Shift (DGCS) It aims to simultaneously detect unknown-class samples and classify known-class samples in the target domains. Compared to prior DG works, we face two new challenges: 1) how to learn the concept of unknown'' during training with only source known-class samples, and 2) how to adapt the source-trained model to unseen environments.
  arXiv  Detail & Related papers  (2023-10-07T07:53:12Z)
• DARE-GRAM : Unsupervised Domain Adaptation Regression by Aligning Inverse Gram Matrices [3.5933327773749513]
  Unlabelled Domain Adaptation Regression (DAR) aims to bridge the domain gap between a labeled source dataset and an unsupervised target dataset for regression problems. We present a different perspective for the DAR problem by analyzing the closed-form ordinary least square(OLS) solution to the linear regressor in the deep domain adaptation context. We propose a simple yet effective DAR method which leverages the pseudo-inverse low-rank property to align the scale and angle in a selected subspace.
  arXiv  Detail & Related papers  (2023-03-23T15:04:23Z)
• Label Alignment Regularization for Distribution Shift [63.228879525056904]
  Recent work has highlighted the label alignment property (LAP) in supervised learning, where the vector of all labels in the dataset is mostly in the span of the top few singular vectors of the data matrix. We propose a regularization method for unsupervised domain adaptation that encourages alignment between the predictions in the target domain and its top singular vectors. We report improved performance over domain adaptation baselines in well-known tasks such as MNIST-USPS domain adaptation and cross-lingual sentiment analysis.
  arXiv  Detail & Related papers  (2022-11-27T22:54:48Z)
• Semi-Supervised Domain Adaptation with Prototypical Alignment and Consistency Learning [86.6929930921905]
  This paper studies how much it can help address domain shifts if we further have a few target samples labeled. To explore the full potential of landmarks, we incorporate a prototypical alignment (PA) module which calculates a target prototype for each class from the landmarks. Specifically, we severely perturb the labeled images, making PA non-trivial to achieve and thus promoting model generalizability.
  arXiv  Detail & Related papers  (2021-04-19T08:46:08Z)
• Latent Space Regularization for Unsupervised Domain Adaptation in Semantic Segmentation [14.050836886292869]
  We introduce feature-level space-shaping regularization strategies to reduce the domain discrepancy in semantic segmentation. We verify the effectiveness of such methods in the autonomous driving setting.
  arXiv  Detail & Related papers  (2021-04-06T16:07:22Z)
• Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation [109.73983088432364]
  We propose the first method that aims to simultaneously learn invariant representations and risks under the setting of semi-supervised domain adaptation (Semi-DA) We introduce the LIRR algorithm for jointly textbfLearning textbfInvariant textbfRepresentations and textbfRisks.
  arXiv  Detail & Related papers  (2020-10-09T15:42:35Z)
• Adversarial Weighting for Domain Adaptation in Regression [4.34858896385326]
  We present a novel instance-based approach to handle regression tasks in the context of supervised domain adaptation. We develop an adversarial network algorithm which learns both the source weighting scheme and the task in one feed-forward gradient descent.
  arXiv  Detail & Related papers  (2020-06-15T09:44:04Z)
• Supervised Domain Adaptation using Graph Embedding [86.3361797111839]
  Domain adaptation methods assume that distributions between the two domains are shifted and attempt to realign them. We propose a generic framework based on graph embedding. We show that the proposed approach leads to a powerful Domain Adaptation framework.
  arXiv  Detail & Related papers  (2020-03-09T12:25:13Z)
• Unsupervised Domain Adaptive Object Detection using Forward-Backward Cyclic Adaptation [13.163271874039191]
  We present a novel approach to perform the unsupervised domain adaptation for object detection through forward-backward cyclic (FBC) training. Recent adversarial training based domain adaptation methods have shown their effectiveness on minimizing domain discrepancy via marginal feature distributions alignment. We propose Forward-Backward Cyclic Adaptation, which iteratively computes adaptation from source to target via backward hopping and from target to source via forward passing.
  arXiv  Detail & Related papers  (2020-02-03T06:24:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.