Label-template based Few-Shot Text Classification with Contrastive Learning

• URL: http://arxiv.org/abs/2412.10110v1
• Date: Fri, 13 Dec 2024 12:51:50 GMT
• Title: Label-template based Few-Shot Text Classification with Contrastive Learning
• Authors: Guanghua Hou, Shuhui Cao, Deqiang Ouyang, Ning Wang,
• Abstract summary: We propose a simple and effective few-shot text classification framework. Label templates are embedded into input sentences to fully utilize the potential value of class labels. supervised contrastive learning is utilized to model the interaction information between support samples and query samples.
• Score: 7.964862748983985
• License:
• Abstract: As an algorithmic framework for learning to learn, meta-learning provides a promising solution for few-shot text classification. However, most existing research fail to give enough attention to class labels. Traditional basic framework building meta-learner based on prototype networks heavily relies on inter-class variance, and it is easily influenced by noise. To address these limitations, we proposes a simple and effective few-shot text classification framework. In particular, the corresponding label templates are embed into input sentences to fully utilize the potential value of class labels, guiding the pre-trained model to generate more discriminative text representations through the semantic information conveyed by labels. With the continuous influence of label semantics, supervised contrastive learning is utilized to model the interaction information between support samples and query samples. Furthermore, the averaging mechanism is replaced with an attention mechanism to highlight vital semantic information. To verify the proposed scheme, four typical datasets are employed to assess the performance of different methods. Experimental results demonstrate that our method achieves substantial performance enhancements and outperforms existing state-of-the-art models on few-shot text classification tasks.

Related papers

• Leveraging Annotator Disagreement for Text Classification [3.6625157427847963]
  It is common practice in text classification to only use one majority label for model training even if a dataset has been annotated by multiple annotators. This paper proposes three strategies to leverage annotator disagreement for text classification: a probability-based multi-label method, an ensemble system, and instruction tuning.
  arXiv  Detail & Related papers  (2024-09-26T06:46:53Z)
• XAI-CLASS: Explanation-Enhanced Text Classification with Extremely Weak Supervision [6.406111099707549]
  XAI-CLASS is a novel explanation-enhanced weakly-supervised text classification method. It incorporates word saliency prediction as an auxiliary task. XAI-CLASS outperforms other weakly-supervised text classification methods significantly.
  arXiv  Detail & Related papers  (2023-10-31T23:24:22Z)
• A Visual Interpretation-Based Self-Improved Classification System Using Virtual Adversarial Training [4.722922834127293]
  This paper proposes a visual interpretation-based self-improving classification model with a combination of virtual adversarial training (VAT) and BERT models to address the problems. Specifically, a fine-tuned BERT model is used as a classifier to classify the sentiment of the text. The predicted sentiment classification labels are used as part of the input of another BERT for spam classification via a semi-supervised training manner.
  arXiv  Detail & Related papers  (2023-09-03T15:07:24Z)
• Description-Enhanced Label Embedding Contrastive Learning for Text Classification [65.01077813330559]
  Self-Supervised Learning (SSL) in model learning process and design a novel self-supervised Relation of Relation (R2) classification task. Relation of Relation Learning Network (R2-Net) for text classification, in which text classification and R2 classification are treated as optimization targets. external knowledge from WordNet to obtain multi-aspect descriptions for label semantic learning.
  arXiv  Detail & Related papers  (2023-06-15T02:19:34Z)
• Learning Context-aware Classifier for Semantic Segmentation [88.88198210948426]
  In this paper, contextual hints are exploited via learning a context-aware classifier. Our method is model-agnostic and can be easily applied to generic segmentation models. With only negligible additional parameters and +2% inference time, decent performance gain has been achieved on both small and large models.
  arXiv  Detail & Related papers  (2023-03-21T07:00:35Z)
• Resolving label uncertainty with implicit posterior models [71.62113762278963]
  We propose a method for jointly inferring labels across a collection of data samples. By implicitly assuming the existence of a generative model for which a differentiable predictor is the posterior, we derive a training objective that allows learning under weak beliefs.
  arXiv  Detail & Related papers  (2022-02-28T18:09:44Z)
• CLLD: Contrastive Learning with Label Distance for Text Classificatioin [0.6299766708197883]
  We propose Contrastive Learning with Label Distance (CLLD) for learning contrastive classes. CLLD ensures the flexibility within the subtle differences that lead to different label assignments. Our experiments suggest that the learned label distance relieve the adversarial nature of interclasses.
  arXiv  Detail & Related papers  (2021-10-25T07:07:14Z)
• Multi-Label Image Classification with Contrastive Learning [57.47567461616912]
  We show that a direct application of contrastive learning can hardly improve in multi-label cases. We propose a novel framework for multi-label classification with contrastive learning in a fully supervised setting.
  arXiv  Detail & Related papers  (2021-07-24T15:00:47Z)
• Improving Classification through Weak Supervision in Context-specific Conversational Agent Development for Teacher Education [1.215785021723604]
  The effort required to develop an educational scenario specific conversational agent is time consuming. Previous approaches to modeling annotations have relied on labeling thousands of examples and calculating inter-annotator agreement and majority votes. We propose using a multi-task weak supervision method combined with active learning to address these concerns.
  arXiv  Detail & Related papers  (2020-10-23T23:39:40Z)
• Dynamic Semantic Matching and Aggregation Network for Few-shot Intent Detection [69.2370349274216]
  Few-shot Intent Detection is challenging due to the scarcity of available annotated utterances. Semantic components are distilled from utterances via multi-head self-attention. Our method provides a comprehensive matching measure to enhance representations of both labeled and unlabeled instances.
  arXiv  Detail & Related papers  (2020-10-06T05:16:38Z)
• Cooperative Bi-path Metric for Few-shot Learning [50.98891758059389]
  We make two contributions to investigate the few-shot classification problem. We report a simple and effective baseline trained on base classes in the way of traditional supervised learning. We propose a cooperative bi-path metric for classification, which leverages the correlations between base classes and novel classes to further improve the accuracy.
  arXiv  Detail & Related papers  (2020-08-10T11:28:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.