Evaluating Unsupervised Text Classification: Zero-shot and Similarity-based Approaches

• URL: http://arxiv.org/abs/2211.16285v1
• Date: Tue, 29 Nov 2022 15:14:47 GMT
• Title: Evaluating Unsupervised Text Classification: Zero-shot and Similarity-based Approaches
• Authors: Tim Schopf, Daniel Braun, Florian Matthes
• Abstract summary: Similarity-based approaches attempt to classify instances based on similarities between text document representations and class description representations. Zero-shot text classification approaches aim to generalize knowledge gained from a training task by assigning appropriate labels of unknown classes to text documents. This paper conducts a systematic evaluation of different similarity-based and zero-shot approaches for text classification of unseen classes.
• Score: 0.6767885381740952
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text classification of unseen classes is a challenging Natural Language Processing task and is mainly attempted using two different types of approaches. Similarity-based approaches attempt to classify instances based on similarities between text document representations and class description representations. Zero-shot text classification approaches aim to generalize knowledge gained from a training task by assigning appropriate labels of unknown classes to text documents. Although existing studies have already investigated individual approaches to these categories, the experiments in literature do not provide a consistent comparison. This paper addresses this gap by conducting a systematic evaluation of different similarity-based and zero-shot approaches for text classification of unseen classes. Different state-of-the-art approaches are benchmarked on four text classification datasets, including a new dataset from the medical domain. Additionally, novel SimCSE and SBERT-based baselines are proposed, as other baselines used in existing work yield weak classification results and are easily outperformed. Finally, the novel similarity-based Lbl2TransformerVec approach is presented, which outperforms previous state-of-the-art approaches in unsupervised text classification. Our experiments show that similarity-based approaches significantly outperform zero-shot approaches in most cases. Additionally, using SimCSE or SBERT embeddings instead of simpler text representations increases similarity-based classification results even further.

Related papers

• Contextuality Helps Representation Learning for Generalized Category Discovery [5.885208652383516]
  This paper introduces a novel approach to Generalized Category Discovery (GCD) by leveraging the concept of contextuality. Our model integrates two levels of contextuality: instance-level, where nearest-neighbor contexts are utilized for contrastive learning, and cluster-level, employing contrastive learning. The integration of the contextual information effectively improves the feature learning and thereby the classification accuracy of all categories.
  arXiv  Detail & Related papers  (2024-07-29T07:30:41Z)
• Detecting Statements in Text: A Domain-Agnostic Few-Shot Solution [1.3654846342364308]
  State-of-the-art approaches usually involve fine-tuning models on large annotated datasets, which are costly to produce. We propose and release a qualitative and versatile few-shot learning methodology as a common paradigm for any claim-based textual classification task. We illustrate this methodology in the context of three tasks: climate change contrarianism detection, topic/stance classification and depression-relates symptoms detection.
  arXiv  Detail & Related papers  (2024-05-09T12:03:38Z)
• XAI-CLASS: Explanation-Enhanced Text Classification with Extremely Weak Supervision [6.406111099707549]
  XAI-CLASS is a novel explanation-enhanced weakly-supervised text classification method. It incorporates word saliency prediction as an auxiliary task. XAI-CLASS outperforms other weakly-supervised text classification methods significantly.
  arXiv  Detail & Related papers  (2023-10-31T23:24:22Z)
• LIME: Weakly-Supervised Text Classification Without Seeds [1.2691047660244335]
  In weakly-supervised text classification, only label names act as sources of supervision. We present LIME, a framework for weakly-supervised text classification. We find that combining weakly-supervised classification and textual entailment mitigates shortcomings of both.
  arXiv  Detail & Related papers  (2022-10-13T04:28:28Z)
• Class-incremental Novel Class Discovery [76.35226130521758]
  We study the new task of class-incremental Novel Class Discovery (class-iNCD) We propose a novel approach for class-iNCD which prevents forgetting of past information about the base classes. Our experiments, conducted on three common benchmarks, demonstrate that our method significantly outperforms state-of-the-art approaches.
  arXiv  Detail & Related papers  (2022-07-18T13:49:27Z)
• Many-Class Text Classification with Matching [65.74328417321738]
  We formulate textbfText textbfClassification as a textbfMatching problem between the text and the labels, and propose a simple yet effective framework named TCM. Compared with previous text classification approaches, TCM takes advantage of the fine-grained semantic information of the classification labels.
  arXiv  Detail & Related papers  (2022-05-23T15:51:19Z)
• Are We Really Making Much Progress in Text Classification? A Comparative Review [2.579878570919875]
  This study reviews and compares methods for single-label and multi-label text classification. Results reveal that all recently proposed graph-based and hierarchy-based methods fail to outperform pre-trained language models.
  arXiv  Detail & Related papers  (2022-04-08T09:28:20Z)
• Out-of-Category Document Identification Using Target-Category Names as Weak Supervision [64.671654559798]
  Out-of-category detection aims to distinguish documents according to their semantic relevance to the inlier (or target) categories. We present an out-of-category detection framework, which effectively measures how confidently each document belongs to one of the target categories.
  arXiv  Detail & Related papers  (2021-11-24T21:01:25Z)
• Category Contrast for Unsupervised Domain Adaptation in Visual Tasks [92.9990560760593]
  We propose a novel Category Contrast technique (CaCo) that introduces semantic priors on top of instance discrimination for visual UDA tasks. CaCo is complementary to existing UDA methods and generalizable to other learning setups such as semi-supervised learning, unsupervised model adaptation, etc.
  arXiv  Detail & Related papers  (2021-06-05T12:51:35Z)
• Binary Classification from Multiple Unlabeled Datasets via Surrogate Set Classification [94.55805516167369]
  We propose a new approach for binary classification from m U-sets for $mge2$. Our key idea is to consider an auxiliary classification task called surrogate set classification (SSC)
  arXiv  Detail & Related papers  (2021-02-01T07:36:38Z)
• Cooperative Bi-path Metric for Few-shot Learning [50.98891758059389]
  We make two contributions to investigate the few-shot classification problem. We report a simple and effective baseline trained on base classes in the way of traditional supervised learning. We propose a cooperative bi-path metric for classification, which leverages the correlations between base classes and novel classes to further improve the accuracy.
  arXiv  Detail & Related papers  (2020-08-10T11:28:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.