Margin Discrepancy-based Adversarial Training for Multi-Domain Text Classification

• URL: http://arxiv.org/abs/2403.00888v1
• Date: Fri, 1 Mar 2024 11:54:14 GMT
• Title: Margin Discrepancy-based Adversarial Training for Multi-Domain Text Classification
• Authors: Yuan Wu
• Abstract summary: Multi-domain text classification (MDTC) endeavors to harness available resources from correlated domains to enhance the classification accuracy of the target domain. Most MDTC approaches that embrace adversarial training and the shared-private paradigm exhibit cutting-edge performance. We propose a margin discrepancy-based adversarial training (MDAT) approach for MDTC, in accordance with our theoretical analysis.
• Score: 6.629561563470492
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-domain text classification (MDTC) endeavors to harness available resources from correlated domains to enhance the classification accuracy of the target domain. Presently, most MDTC approaches that embrace adversarial training and the shared-private paradigm exhibit cutting-edge performance. Unfortunately, these methods face a non-negligible challenge: the absence of theoretical guarantees in the design of MDTC algorithms. The dearth of theoretical underpinning poses a substantial impediment to the advancement of MDTC algorithms. To tackle this problem, we first provide a theoretical analysis of MDTC by decomposing the MDTC task into multiple domain adaptation tasks. We incorporate the margin discrepancy as the measure of domain divergence and establish a new generalization bound based on Rademacher complexity. Subsequently, we propose a margin discrepancy-based adversarial training (MDAT) approach for MDTC, in accordance with our theoretical analysis. To validate the efficacy of the proposed MDAT method, we conduct empirical studies on two MDTC benchmarks. The experimental results demonstrate that our MDAT approach surpasses state-of-the-art baselines on both datasets.

Related papers

• Multi-Agent Reinforcement Learning from Human Feedback: Data Coverage and Algorithmic Techniques [65.55451717632317]
  We study Multi-Agent Reinforcement Learning from Human Feedback (MARLHF), exploring both theoretical foundations and empirical validations. We define the task as identifying Nash equilibrium from a preference-only offline dataset in general-sum games. Our findings underscore the multifaceted approach required for MARLHF, paving the way for effective preference-based multi-agent systems.
  arXiv  Detail & Related papers  (2024-09-01T13:14:41Z)
• Top-K Pairwise Ranking: Bridging the Gap Among Ranking-Based Measures for Multi-Label Classification [120.37051160567277]
  This paper proposes a novel measure named Top-K Pairwise Ranking (TKPR) A series of analyses show that TKPR is compatible with existing ranking-based measures. On the other hand, we establish a sharp generalization bound for the proposed framework based on a novel technique named data-dependent contraction.
  arXiv  Detail & Related papers  (2024-07-09T09:36:37Z)
• A Generalization Theory of Cross-Modality Distillation with Contrastive Learning [49.35244441141323]
  Cross-modality distillation arises as an important topic for data modalities containing limited knowledge. We formulate a general framework of cross-modality contrastive distillation (CMCD), built upon contrastive learning. Our algorithm outperforms existing algorithms consistently by a margin of 2-3% across diverse modalities and tasks.
  arXiv  Detail & Related papers  (2024-05-06T11:05:13Z)
• Regularized Conditional Alignment for Multi-Domain Text Classification [6.629561563470492]
  We propose a method called Regularized Conditional Alignment (RCA) to align the joint distributions of domains and classes. We employ entropy minimization and virtual adversarial training to constrain the uncertainty of predictions pertaining to unlabeled data. Empirical results on two benchmark datasets demonstrate that our RCA approach outperforms state-of-the-art MDTC techniques.
  arXiv  Detail & Related papers  (2023-12-18T05:52:05Z)
• MADG: Margin-based Adversarial Learning for Domain Generalization [25.45950080930517]
  We propose a novel adversarial learning DG algorithm, MADG, motivated by a margin loss-based discrepancy metric. The proposed MADG model learns domain-invariant features across all source domains and uses adversarial training to generalize well to the unseen target domain. We extensively experiment with the MADG model on popular real-world DG datasets.
  arXiv  Detail & Related papers  (2023-11-14T19:53:09Z)
• Algorithm-Dependent Bounds for Representation Learning of Multi-Source Domain Adaptation [7.6249291891777915]
  We use information-theoretic tools to derive a novel analysis of Multi-source Domain Adaptation (MDA) from the representation learning perspective. We propose a novel deep MDA algorithm, implicitly addressing the target shift through joint alignment. The proposed algorithm has comparable performance to the state-of-the-art on target-shifted MDA benchmark with improved memory efficiency.
  arXiv  Detail & Related papers  (2023-04-04T18:32:20Z)
• Making Linear MDPs Practical via Contrastive Representation Learning [101.75885788118131]
  It is common to address the curse of dimensionality in Markov decision processes (MDPs) by exploiting low-rank representations. We consider an alternative definition of linear MDPs that automatically ensures normalization while allowing efficient representation learning. We demonstrate superior performance over existing state-of-the-art model-based and model-free algorithms on several benchmarks.
  arXiv  Detail & Related papers  (2022-07-14T18:18:02Z)
• On Certifying and Improving Generalization to Unseen Domains [87.00662852876177]
  Domain Generalization aims to learn models whose performance remains high on unseen domains encountered at test-time. It is challenging to evaluate DG algorithms comprehensively using a few benchmark datasets. We propose a universal certification framework that can efficiently certify the worst-case performance of any DG method.
  arXiv  Detail & Related papers  (2022-06-24T16:29:43Z)
• Deep End-to-end Causal Inference [36.822180456180675]
  Causal inference is essential for data-driven decision making across domains such as business engagement, medical treatment or policy making. We develop Deep End-to-end Causal Inference (DECI), a single flow-based method that takes in observational data and can perform both causal discovery and inference. Our results show the superior performance of DECI when compared to relevant baselines for both causal discovery and (C)ATE estimation.
  arXiv  Detail & Related papers  (2022-02-04T15:40:28Z)
• Co-Regularized Adversarial Learning for Multi-Domain Text Classification [19.393393465837377]
  Multi-domain text classification (MDTC) aims to leverage all available resources from multiple domains to learn a predictive model that can generalize well on these domains. Recently, many MDTC methods adopt adversarial learning, shared-private paradigm, and entropy minimization to yield state-of-the-art results. These approaches face three issues: (1) Minimizing domain divergence can not fully guarantee the success of domain alignment; (2) Aligning marginal feature distributions can not fully guarantee the discriminability of the learned features; and (3) Standard entropy minimization may make the predictions on unlabeled data over-confident, deteriorating the disc
  arXiv  Detail & Related papers  (2022-01-30T12:15:41Z)
• Model-Based Domain Generalization [96.84818110323518]
  We propose a novel approach for the domain generalization problem called Model-Based Domain Generalization. Our algorithms beat the current state-of-the-art methods on the very-recently-proposed WILDS benchmark by up to 20 percentage points.
  arXiv  Detail & Related papers  (2021-02-23T00:59:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.