PESCO: Prompt-enhanced Self Contrastive Learning for Zero-shot Text Classification

• URL: http://arxiv.org/abs/2305.14963v1
• Date: Wed, 24 May 2023 09:57:06 GMT
• Title: PESCO: Prompt-enhanced Self Contrastive Learning for Zero-shot Text Classification
• Authors: Yau-Shian Wang and Ta-Chung Chi and Ruohong Zhang and Yiming Yang
• Abstract summary: PESCO is a contrastive learning framework that substantially improves the performance of zero-shot text classification. PESCO achieves state-of-the-art performance on four benchmark text classification datasets.
• Score: 32.02762416063338
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We present PESCO, a novel contrastive learning framework that substantially improves the performance of zero-shot text classification. We formulate text classification as a neural text matching problem where each document is treated as a query, and the system learns the mapping from each query to the relevant class labels by (1) adding prompts to enhance label matching, and (2) using retrieved labels to enrich the training set in a self-training loop of contrastive learning. PESCO achieves state-of-the-art performance on four benchmark text classification datasets. On DBpedia, we achieve 98.5\% accuracy without any labeled data, which is close to the fully-supervised result. Extensive experiments and analyses show all the components of PESCO are necessary for improving the performance of zero-shot text classification.

Related papers

• Gen-Z: Generative Zero-Shot Text Classification with Contextualized Label Descriptions [50.92702206798324]
  We propose a generative prompting framework for zero-shot text classification. GEN-Z measures the LM likelihood of input text conditioned on natural language descriptions of labels. We show that zero-shot classification with simple contextualization of the data source consistently outperforms both zero-shot and few-shot baselines.
  arXiv  Detail & Related papers  (2023-11-13T07:12:57Z)
• TextFormer: A Query-based End-to-End Text Spotter with Mixed Supervision [61.186488081379]
  We propose TextFormer, a query-based end-to-end text spotter with Transformer architecture. TextFormer builds upon an image encoder and a text decoder to learn a joint semantic understanding for multi-task modeling. It allows for mutual training and optimization of classification, segmentation, and recognition branches, resulting in deeper feature sharing.
  arXiv  Detail & Related papers  (2023-06-06T03:37:41Z)
• Label Agnostic Pre-training for Zero-shot Text Classification [4.9081735096855565]
  In real-world applications, there exists an infinite label space for describing a given text. We introduce two new simple yet effective pre-training strategies, Implicit and Explicit pre-training. These methods inject aspect-level understanding into the model at train time with the goal of conditioning the model to build task-level understanding.
  arXiv  Detail & Related papers  (2023-05-25T22:55:32Z)
• The Benefits of Label-Description Training for Zero-Shot Text Classification [35.27224341685012]
  Pretrained language models have improved zero-shot text classification. We propose a simple way to further improve zero-shot accuracies with minimal effort.
  arXiv  Detail & Related papers  (2023-05-03T16:19:31Z)
• Task-Specific Embeddings for Ante-Hoc Explainable Text Classification [6.671252951387647]
  We propose an alternative training objective in which we learn task-specific embeddings of text. Our proposed objective learns embeddings such that all texts that share the same target class label should be close together. We present extensive experiments which show that the benefits of ante-hoc explainability and incremental learning come at no cost in overall classification accuracy.
  arXiv  Detail & Related papers  (2022-11-30T19:56:25Z)
• Many-Class Text Classification with Matching [65.74328417321738]
  We formulate textbfText textbfClassification as a textbfMatching problem between the text and the labels, and propose a simple yet effective framework named TCM. Compared with previous text classification approaches, TCM takes advantage of the fine-grained semantic information of the classification labels.
  arXiv  Detail & Related papers  (2022-05-23T15:51:19Z)
• Label Semantic Aware Pre-training for Few-shot Text Classification [53.80908620663974]
  We propose Label Semantic Aware Pre-training (LSAP) to improve the generalization and data efficiency of text classification systems. LSAP incorporates label semantics into pre-trained generative models (T5 in our case) by performing secondary pre-training on labeled sentences from a variety of domains.
  arXiv  Detail & Related papers  (2022-04-14T17:33:34Z)
• Minimally-Supervised Structure-Rich Text Categorization via Learning on Text-Rich Networks [61.23408995934415]
  We propose a novel framework for minimally supervised categorization by learning from the text-rich network. Specifically, we jointly train two modules with different inductive biases -- a text analysis module for text understanding and a network learning module for class-discriminative, scalable network learning. Our experiments show that given only three seed documents per category, our framework can achieve an accuracy of about 92%.
  arXiv  Detail & Related papers  (2021-02-23T04:14:34Z)
• Unsupervised Label Refinement Improves Dataless Text Classification [48.031421660674745]
  Dataless text classification is capable of classifying documents into previously unseen labels by assigning a score to any document paired with a label description. While promising, it crucially relies on accurate descriptions of the label set for each downstream task. This reliance causes dataless classifiers to be highly sensitive to the choice of label descriptions and hinders the broader application of dataless classification in practice.
  arXiv  Detail & Related papers  (2020-12-08T03:37:50Z)
• Description Based Text Classification with Reinforcement Learning [34.18824470728299]
  We propose a new framework for text classification, in which each category label is associated with a category description. We observe significant performance boosts over strong baselines on a wide range of text classification tasks.
  arXiv  Detail & Related papers  (2020-02-08T02:14:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.