Dual Contrastive Learning: Text Classification via Label-Aware Data Augmentation

• URL: http://arxiv.org/abs/2201.08702v1
• Date: Fri, 21 Jan 2022 13:59:45 GMT
• Title: Dual Contrastive Learning: Text Classification via Label-Aware Data Augmentation
• Abstract summary: We introduce a dual contrastive learning framework that simultaneously learns the features of input samples and the parameters of classifiers in the same space. Specifically, DualCL regards the parameters of the classifiers as augmented samples associating to different labels and then exploits the contrastive learning between the input samples and the augmented samples. Empirical studies on five benchmark text classification datasets demonstrate the improvement in classification accuracy and confirm the capability of learning discriminative representations of DualCL.
• Score: 38.61781713469419
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Contrastive learning has achieved remarkable success in representation learning via self-supervision in unsupervised settings. However, effectively adapting contrastive learning to supervised learning tasks remains as a challenge in practice. In this work, we introduce a dual contrastive learning (DualCL) framework that simultaneously learns the features of input samples and the parameters of classifiers in the same space. Specifically, DualCL regards the parameters of the classifiers as augmented samples associating to different labels and then exploits the contrastive learning between the input samples and the augmented samples. Empirical studies on five benchmark text classification datasets and their low-resource version demonstrate the improvement in classification accuracy and confirm the capability of learning discriminative representations of DualCL.

Related papers

• ProtoFair: Fair Self-Supervised Contrastive Learning via Pseudo-Counterfactual Pairs [5.192293017737767]
  Existing fairness-aware methods address this by redesigning the self-supervised objective itself.<n>We propose ProtoFair, a fairness-aware contrastive loss designed to work alongside existing SSL objectives without modifying them.<n>ProtoFair integrates seamlessly with both SimCLR and SupCon.
  arXiv  Detail & Related papers  (2026-05-03T17:07:54Z)
• ZeroDiff++: Substantial Unseen Visual-semantic Correlation in Zero-shot Learning [42.855022683292276]
  We introduce metrics to quantify spuriousness for seen and unseen classes.<n>We propose ZeroDiff++, a diffusion-based generative framework.<n>Experiments on three ZSL benchmarks demonstrate that ZeroDiff++ not only achieves significant improvements over existing ZSL methods but also maintains robust performance even with scarce training data.
  arXiv  Detail & Related papers  (2026-02-12T20:52:07Z)
• Beyond Words: Augmenting Discriminative Richness via Diffusions in Unsupervised Prompt Learning [23.129998055266245]
  Current pseudo-labeling strategies often struggle with mismatches between semantic and visual information. We introduce a simple yet effective approach called textbfAugmenting Dtextbfiscriminative textbfRichness via Diffusions (AiR)
  arXiv  Detail & Related papers  (2025-04-16T10:09:45Z)
• Semantic Retrieval Augmented Contrastive Learning for Sequential Recommendation [17.18176550968383]
  We propose a novel approach named Semantic Retrieval Augmented Contrastive Learning (SRA-CL), which leverages semantic information to improve the reliability of contrastive samples. SRA-CL comprises two main components: (1) Cross-Sequence Contrastive Learning via User Semantic Retrieval, which utilizes large language models (LLMs) to understand diverse user preferences and retrieve semantically similar users to form reliable positive samples through a learnable sample method; and (2) Intra-Sequence Contrastive Learning via Item Semantic Retrieval, which employs LLMs to comprehend items and retrieve similar items to perform semantic-based item substitution
  arXiv  Detail & Related papers  (2025-03-06T07:25:19Z)
• Logit Separability-Driven Samples and Multiple Class-Related Words Selection for Advancing In-Context Learning [0.0]
  We introduce logit separability, a criterion to assess the clarity of both samples and class-related words at the logit level. We find that incorporating multiple class-related words for each sample, rather than relying on a single class name, improves performance by offering a broader range of label information. We propose LICL, a logit separability-based method that jointly organizes samples and integrates multiple class-related words into each sample-label pair.
  arXiv  Detail & Related papers  (2024-06-16T12:11:46Z)
• Channel-Wise Contrastive Learning for Learning with Noisy Labels [60.46434734808148]
  We introduce channel-wise contrastive learning (CWCL) to distinguish authentic label information from noise. Unlike conventional instance-wise contrastive learning (IWCL), CWCL tends to yield more nuanced and resilient features aligned with the authentic labels. Our strategy is twofold: firstly, using CWCL to extract pertinent features to identify cleanly labeled samples, and secondly, progressively fine-tuning using these samples.
  arXiv  Detail & Related papers  (2023-08-14T06:04:50Z)
• Cluster-aware Contrastive Learning for Unsupervised Out-of-distribution Detection [0.0]
  Unsupervised out-of-distribution (OOD) Detection aims to separate the samples falling outside the distribution of training data without label information. We propose Cluster-aware Contrastive Learning (CCL) framework for unsupervised OOD detection, which considers both instance-level and semantic-level information.
  arXiv  Detail & Related papers  (2023-02-06T07:21:03Z)
• Non-Contrastive Learning Meets Language-Image Pre-Training [145.6671909437841]
  We study the validity of non-contrastive language-image pre-training (nCLIP) We introduce xCLIP, a multi-tasking framework combining CLIP and nCLIP, and show that nCLIP aids CLIP in enhancing feature semantics.
  arXiv  Detail & Related papers  (2022-10-17T17:57:46Z)
• Adaptive Soft Contrastive Learning [19.45520684918576]
  This paper proposes an adaptive method that introduces soft inter-sample relations, namely Adaptive Soft Contrastive Learning (ASCL) As an effective and concise plug-in module for existing self-supervised learning frameworks, ASCL achieves the best performance on several benchmarks.
  arXiv  Detail & Related papers  (2022-07-22T16:01:07Z)
• Using Representation Expressiveness and Learnability to Evaluate Self-Supervised Learning Methods [61.49061000562676]
  We introduce Cluster Learnability (CL) to assess learnability. CL is measured in terms of the performance of a KNN trained to predict labels obtained by clustering the representations with K-means. We find that CL better correlates with in-distribution model performance than other competing recent evaluation schemes.
  arXiv  Detail & Related papers  (2022-06-02T19:05:13Z)
• CLLD: Contrastive Learning with Label Distance for Text Classificatioin [0.6299766708197883]
  We propose Contrastive Learning with Label Distance (CLLD) for learning contrastive classes. CLLD ensures the flexibility within the subtle differences that lead to different label assignments. Our experiments suggest that the learned label distance relieve the adversarial nature of interclasses.
  arXiv  Detail & Related papers  (2021-10-25T07:07:14Z)
• Neighborhood Contrastive Learning for Novel Class Discovery [79.14767688903028]
  We build a new framework, named Neighborhood Contrastive Learning, to learn discriminative representations that are important to clustering performance. We experimentally demonstrate that these two ingredients significantly contribute to clustering performance and lead our model to outperform state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2021-06-20T17:34:55Z)
• Visual Transformer for Task-aware Active Learning [49.903358393660724]
  We present a novel pipeline for pool-based Active Learning. Our method exploits accessible unlabelled examples during training to estimate their co-relation with the labelled examples. Visual Transformer models non-local visual concept dependency between labelled and unlabelled examples.
  arXiv  Detail & Related papers  (2021-06-07T17:13:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.