Description Boosting for Zero-Shot Entity and Relation Classification

• URL: http://arxiv.org/abs/2406.02245v1
• Date: Tue, 4 Jun 2024 12:09:44 GMT
• Title: Description Boosting for Zero-Shot Entity and Relation Classification
• Authors: Gabriele Picco, Leopold Fuchs, Marcos Martínez Galindo, Alberto Purpura, Vanessa López, Hoang Thanh Lam,
• Abstract summary: We show that Zero-Shot Learning (ZSL) methods are sensitive to provided textual descriptions of entities (or relations) We propose a strategy for generating variations of an initial description and an ensemble method capable of boosting the predictions of zero-shot models through description enhancement.
• Score: 5.8959034854546815
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Zero-shot entity and relation classification models leverage available external information of unseen classes -- e.g., textual descriptions -- to annotate input text data. Thanks to the minimum data requirement, Zero-Shot Learning (ZSL) methods have high value in practice, especially in applications where labeled data is scarce. Even though recent research in ZSL has demonstrated significant results, our analysis reveals that those methods are sensitive to provided textual descriptions of entities (or relations). Even a minor modification of descriptions can lead to a change in the decision boundary between entity (or relation) classes. In this paper, we formally define the problem of identifying effective descriptions for zero shot inference. We propose a strategy for generating variations of an initial description, a heuristic for ranking them and an ensemble method capable of boosting the predictions of zero-shot models through description enhancement. Empirical results on four different entity and relation classification datasets show that our proposed method outperform existing approaches and achieve new SOTA results on these datasets under the ZSL settings. The source code of the proposed solutions and the evaluation framework are open-sourced.

Related papers

• Refining Sentence Embedding Model through Ranking Sentences Generation with Large Language Models [60.00178316095646]
  Sentence embedding is essential for many NLP tasks, with contrastive learning methods achieving strong performance using datasets like NLI. Recent studies leverage large language models (LLMs) to generate sentence pairs, reducing annotation dependency. We propose a method for controlling the generation direction of LLMs in the latent space. Unlike unconstrained generation, the controlled approach ensures meaningful semantic divergence. Experiments on multiple benchmarks demonstrate that our method achieves new SOTA performance with a modest cost in ranking sentence synthesis.
  arXiv  Detail & Related papers  (2025-02-19T12:07:53Z)
• Unleashing the Power of Data Tsunami: A Comprehensive Survey on Data Assessment and Selection for Instruction Tuning of Language Models [33.488331159912136]
  Instruction tuning plays a critical role in aligning large language models (LLMs) with human preference. Data assessment and selection methods have been proposed in the fields of natural language processing (NLP) and deep learning. We present a comprehensive review on existing literature of data assessment and selection especially for instruction tuning of LLMs.
  arXiv  Detail & Related papers  (2024-08-04T16:50:07Z)
• Enhancing Text Classification through LLM-Driven Active Learning and Human Annotation [2.0411082897313984]
  This study introduces a novel methodology that integrates human annotators and Large Language Models. The proposed framework integrates human annotation with the output of LLMs, depending on the model uncertainty levels. The empirical results show a substantial decrease in the costs associated with data annotation while either maintaining or improving model accuracy.
  arXiv  Detail & Related papers  (2024-06-17T21:45:48Z)
• Gen-Z: Generative Zero-Shot Text Classification with Contextualized Label Descriptions [50.92702206798324]
  We propose a generative prompting framework for zero-shot text classification. GEN-Z measures the LM likelihood of input text conditioned on natural language descriptions of labels. We show that zero-shot classification with simple contextualization of the data source consistently outperforms both zero-shot and few-shot baselines.
  arXiv  Detail & Related papers  (2023-11-13T07:12:57Z)
• XAL: EXplainable Active Learning Makes Classifiers Better Low-resource Learners [71.8257151788923]
  We propose a novel Explainable Active Learning framework (XAL) for low-resource text classification. XAL encourages classifiers to justify their inferences and delve into unlabeled data for which they cannot provide reasonable explanations. Experiments on six datasets show that XAL achieves consistent improvement over 9 strong baselines.
  arXiv  Detail & Related papers  (2023-10-09T08:07:04Z)
• Enlarging Instance-specific and Class-specific Information for Open-set Action Recognition [47.69171542776917]
  We find that features with richer semantic diversity can significantly improve the open-set performance under the same uncertainty scores. A novel Prototypical Similarity Learning (PSL) framework is proposed to keep the instance variance within the same class to retain more IS information.
  arXiv  Detail & Related papers  (2023-03-25T04:07:36Z)
• Active Learning for Abstractive Text Summarization [50.79416783266641]
  We propose the first effective query strategy for Active Learning in abstractive text summarization. We show that using our strategy in AL annotation helps to improve the model performance in terms of ROUGE and consistency scores.
  arXiv  Detail & Related papers  (2023-01-09T10:33:14Z)
• An Additive Instance-Wise Approach to Multi-class Model Interpretation [53.87578024052922]
  Interpretable machine learning offers insights into what factors drive a certain prediction of a black-box system. Existing methods mainly focus on selecting explanatory input features, which follow either locally additive or instance-wise approaches. This work exploits the strengths of both methods and proposes a global framework for learning local explanations simultaneously for multiple target classes.
  arXiv  Detail & Related papers  (2022-07-07T06:50:27Z)
• Zero-shot Learning with Class Description Regularization [10.739164530098755]
  We introduce a novel form of regularization that encourages generative ZSL models to pay more attention to the description of each category. Our empirical results demonstrate improvements over the performance of multiple state-of-the-art models on the task of generalized zero-shot recognition and classification.
  arXiv  Detail & Related papers  (2021-06-30T14:56:15Z)
• Revisiting Few-shot Relation Classification: Evaluation Data and Classification Schemes [57.34346419239118]
  We propose a novel methodology for deriving more realistic few-shot test data from available datasets for supervised RC. This yields a new challenging benchmark for FSL RC, on which state of the art models show poor performance. We propose a novel classification scheme, in which the NOTA category is represented as learned vectors.
  arXiv  Detail & Related papers  (2021-04-17T08:16:49Z)
• Valid Explanations for Learning to Rank Models [5.320400771224103]
  We propose a model agnostic local explanation method that seeks to identify a small subset of input features as explanation to a ranking decision. We introduce new notions of validity and completeness of explanations specifically for rankings, based on the presence or absence of selected features.
  arXiv  Detail & Related papers  (2020-04-29T06:21:56Z)
• Proposal Learning for Semi-Supervised Object Detection [76.83284279733722]
  It is non-trivial to train object detectors on unlabeled data due to the unavailability of ground truth labels. We present a proposal learning approach to learn proposal features and predictions from both labeled and unlabeled data.
  arXiv  Detail & Related papers  (2020-01-15T00:06:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.