ProTeCt: Prompt Tuning for Taxonomic Open Set Classification

• URL: http://arxiv.org/abs/2306.02240v2
• Date: Thu, 28 Mar 2024 05:35:46 GMT
• Title: ProTeCt: Prompt Tuning for Taxonomic Open Set Classification
• Authors: Tz-Ying Wu, Chih-Hui Ho, Nuno Vasconcelos,
• Abstract summary: Few-shot adaptation methods do not fare well in the taxonomic open set (TOS) setting. We propose a prompt tuning technique that calibrates the hierarchical consistency of model predictions. A new Prompt Tuning for Hierarchical Consistency (ProTeCt) technique is then proposed to calibrate classification across label set granularities.
• Score: 59.59442518849203
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Visual-language foundation models, like CLIP, learn generalized representations that enable zero-shot open-set classification. Few-shot adaptation methods, based on prompt tuning, have been shown to further improve performance on downstream datasets. However, these methods do not fare well in the taxonomic open set (TOS) setting, where the classifier is asked to make predictions from label sets across different levels of semantic granularity. Frequently, they infer incorrect labels at coarser taxonomic class levels, even when the inference at the leaf level (original class labels) is correct. To address this problem, we propose a prompt tuning technique that calibrates the hierarchical consistency of model predictions. A set of metrics of hierarchical consistency, the Hierarchical Consistent Accuracy (HCA) and the Mean Treecut Accuracy (MTA), are first proposed to evaluate TOS model performance. A new Prompt Tuning for Hierarchical Consistency (ProTeCt) technique is then proposed to calibrate classification across label set granularities. Results show that ProTeCt can be combined with existing prompt tuning methods to significantly improve TOS classification without degrading the leaf level classification performance.

Related papers

• Just Leaf It: Accelerating Diffusion Classifiers with Hierarchical Class Pruning [8.209660505275872]
  We present a Hierarchical Diffusion (HDC) that exploits the inherent hierarchical label structure of a dataset. HDC can accelerate inference by up to 60% while maintaining and, in some cases, improving classification accuracy. Our work enables a new control mechanism of the trade-off between speed and precision, making diffusion-based classification more viable for real-world applications.
  arXiv  Detail & Related papers  (2024-11-18T21:34:05Z)
• Prompt Tuned Embedding Classification for Multi-Label Industry Sector Allocation [2.024620791810963]
  This study benchmarks the performance of Prompt Tuning and baselines for multi-label text classification. It is applied to classifying companies into an investment firm's proprietary industry taxonomy. We confirm that the model's performance is consistent across both well-known and less-known companies.
  arXiv  Detail & Related papers  (2023-09-21T13:45:32Z)
• M-Tuning: Prompt Tuning with Mitigated Label Bias in Open-Set Scenarios [103.6153593636399]
  We propose a vision-language prompt tuning method with mitigated label bias (M-Tuning) It introduces open words from the WordNet to extend the range of words forming the prompt texts from only closed-set label words to more, and thus prompts are tuned in a simulated open-set scenario. Our method achieves the best performance on datasets with various scales, and extensive ablation studies also validate its effectiveness.
  arXiv  Detail & Related papers  (2023-03-09T09:05:47Z)
• CHiLS: Zero-Shot Image Classification with Hierarchical Label Sets [24.868024094095983]
  Open vocabulary models (e.g. CLIP) have shown strong performance on zero-shot classification. We propose Classification with Hierarchical Label Sets (or CHiLS) for datasets with implicit semantic hierarchies. CHiLS is simple to implement within existing zero-shot pipelines and requires no additional training cost.
  arXiv  Detail & Related papers  (2023-02-06T03:59:15Z)
• Self-Adaptive Label Augmentation for Semi-supervised Few-shot Classification [121.63992191386502]
  Few-shot classification aims to learn a model that can generalize well to new tasks when only a few labeled samples are available. We propose a semi-supervised few-shot classification method that assigns an appropriate label to each unlabeled sample by a manually defined metric. A major novelty of SALA is the task-adaptive metric, which can learn the metric adaptively for different tasks in an end-to-end fashion.
  arXiv  Detail & Related papers  (2022-06-16T13:14:03Z)
• Cluster & Tune: Boost Cold Start Performance in Text Classification [21.957605438780224]
  In real-world scenarios, a text classification task often begins with a cold start, when labeled data is scarce. We suggest a method to boost the performance of such models by adding an intermediate unsupervised classification task.
  arXiv  Detail & Related papers  (2022-03-20T15:29:34Z)
• Prototypical Classifier for Robust Class-Imbalanced Learning [64.96088324684683]
  We propose textitPrototypical, which does not require fitting additional parameters given the embedding network. Prototypical produces balanced and comparable predictions for all classes even though the training set is class-imbalanced. We test our method on CIFAR-10LT, CIFAR-100LT and Webvision datasets, observing that Prototypical obtains substaintial improvements compared with state of the arts.
  arXiv  Detail & Related papers  (2021-10-22T01:55:01Z)
• SCARF: Self-Supervised Contrastive Learning using Random Feature Corruption [72.35532598131176]
  We propose SCARF, a technique for contrastive learning, where views are formed by corrupting a random subset of features. We show that SCARF complements existing strategies and outperforms alternatives like autoencoders.
  arXiv  Detail & Related papers  (2021-06-29T08:08:33Z)
• Pre-training Is (Almost) All You Need: An Application to Commonsense Reasoning [61.32992639292889]
  Fine-tuning of pre-trained transformer models has become the standard approach for solving common NLP tasks. We introduce a new scoring method that casts a plausibility ranking task in a full-text format. We show that our method provides a much more stable training phase across random restarts.
  arXiv  Detail & Related papers  (2020-04-29T10:54:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.