Exploring Description-Augmented Dataless Intent Classification

• URL: http://arxiv.org/abs/2407.17862v1
• Date: Thu, 25 Jul 2024 08:31:57 GMT
• Title: Exploring Description-Augmented Dataless Intent Classification
• Authors: Ruoyu Hu, Foaad Khosmood, Abbas Edalat,
• Abstract summary: We introduce several schemes to leverage description-augmented embedding similarity for dataless intent classification. We show promising results for dataless classification scaling to a large number of unseen intents.
• Score: 1.5839621757142595
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In this work, we introduce several schemes to leverage description-augmented embedding similarity for dataless intent classification using current state-of-the-art (SOTA) text embedding models. We report results of our methods on four commonly used intent classification datasets and compare against previous works of a similar nature. Our work shows promising results for dataless classification scaling to a large number of unseen intents. We show competitive results and significant improvements (+6.12\% Avg.) over strong zero-shot baselines, all without training on labelled or task-specific data. Furthermore, we provide qualitative error analysis of the shortfalls of this methodology to help guide future research in this area.

Related papers

• Pattern-Based Graph Classification: Comparison of Quality Measures and Importance of Preprocessing [3.1970244655208306]
  Graph classification aims to categorize graphs based on their structural and attribute features, with applications in diverse fields such as social network analysis and bioinformatics.<n>To identify meaningful patterns, a standard approach is to use a quality measure, i.e. a function that evaluates the discriminative power of each pattern.<n>Only a handful of surveys try to provide some insight by comparing these measures, and none of them specifically focuses on graphs.<n>We present a comparative analysis of 38 quality measures from the literature, and propose a method to elaborate a gold standard ranking of the patterns.
  arXiv  Detail & Related papers  (2025-06-19T07:28:41Z)
• Contextuality Helps Representation Learning for Generalized Category Discovery [5.885208652383516]
  This paper introduces a novel approach to Generalized Category Discovery (GCD) by leveraging the concept of contextuality. Our model integrates two levels of contextuality: instance-level, where nearest-neighbor contexts are utilized for contrastive learning, and cluster-level, employing contrastive learning. The integration of the contextual information effectively improves the feature learning and thereby the classification accuracy of all categories.
  arXiv  Detail & Related papers  (2024-07-29T07:30:41Z)
• Towards Weakly-Supervised Hate Speech Classification Across Datasets [47.101942709219784]
  We show the effectiveness of a state-of-the-art weakly-supervised text classification model in various in-dataset and cross-dataset settings. We also conduct an in-depth quantitative and qualitative analysis of the source of poor generalizability of HS classification models.
  arXiv  Detail & Related papers  (2023-05-04T08:15:40Z)
• Evaluating Unsupervised Text Classification: Zero-shot and Similarity-based Approaches [0.6767885381740952]
  Similarity-based approaches attempt to classify instances based on similarities between text document representations and class description representations. Zero-shot text classification approaches aim to generalize knowledge gained from a training task by assigning appropriate labels of unknown classes to text documents. This paper conducts a systematic evaluation of different similarity-based and zero-shot approaches for text classification of unseen classes.
  arXiv  Detail & Related papers  (2022-11-29T15:14:47Z)
• Association Graph Learning for Multi-Task Classification with Category Shifts [68.58829338426712]
  We focus on multi-task classification, where related classification tasks share the same label space and are learned simultaneously. We learn an association graph to transfer knowledge among tasks for missing classes. Our method consistently performs better than representative baselines.
  arXiv  Detail & Related papers  (2022-10-10T12:37:41Z)
• Few-shot Text Classification with Dual Contrastive Consistency [31.141350717029358]
  In this paper, we explore how to utilize pre-trained language model to perform few-shot text classification. We adopt supervised contrastive learning on few labeled data and consistency-regularization on vast unlabeled data.
  arXiv  Detail & Related papers  (2022-09-29T19:26:23Z)
• Annotation Error Detection: Analyzing the Past and Present for a More Coherent Future [63.99570204416711]
  We reimplement 18 methods for detecting potential annotation errors and evaluate them on 9 English datasets. We define a uniform evaluation setup including a new formalization of the annotation error detection task. We release our datasets and implementations in an easy-to-use and open source software package.
  arXiv  Detail & Related papers  (2022-06-05T22:31:45Z)
• AutoNovel: Automatically Discovering and Learning Novel Visual Categories [138.80332861066287]
  We present a new approach called AutoNovel to tackle the problem of discovering novel classes in an image collection given labelled examples of other classes. We evaluate AutoNovel on standard classification benchmarks and substantially outperform current methods for novel category discovery.
  arXiv  Detail & Related papers  (2021-06-29T11:12:16Z)
• Multitask Learning for Class-Imbalanced Discourse Classification [74.41900374452472]
  We show that a multitask approach can improve 7% Micro F1-score upon current state-of-the-art benchmarks. We also offer a comparative review of additional techniques proposed to address resource-poor problems in NLP.
  arXiv  Detail & Related papers  (2021-01-02T07:13:41Z)
• On the Ambiguity of Rank-Based Evaluation of Entity Alignment or Link Prediction Methods [27.27230441498167]
  We take a closer look at the evaluation of two families of methods for enriching information from knowledge graphs: Link Prediction and Entity Alignment. In particular, we demonstrate that all existing scores can hardly be used to compare results across different datasets. We show that this leads to various problems in the interpretation of results, which may support misleading conclusions.
  arXiv  Detail & Related papers  (2020-02-17T12:26:14Z)
• Automatically Discovering and Learning New Visual Categories with Ranking Statistics [145.89790963544314]
  We tackle the problem of discovering novel classes in an image collection given labelled examples of other classes. We learn a general-purpose clustering model and use the latter to identify the new classes in the unlabelled data. We evaluate our approach on standard classification benchmarks and outperform current methods for novel category discovery by a significant margin.
  arXiv  Detail & Related papers  (2020-02-13T18:53:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.