Enhancing Automatic Keyphrase Labelling with Text-to-Text Transfer Transformer (T5) Architecture: A Framework for Keyphrase Generation and Filtering

• URL: http://arxiv.org/abs/2409.16760v1
• Date: Wed, 25 Sep 2024 09:16:46 GMT
• Title: Enhancing Automatic Keyphrase Labelling with Text-to-Text Transfer Transformer (T5) Architecture: A Framework for Keyphrase Generation and Filtering
• Authors: Jorge Gabín, M. Eduardo Ares, Javier Parapar,
• Abstract summary: This paper presents a keyphrase generation model based on the Text-to-Text Transfer Transformer (T5) architecture. We also present a novel keyphrase filtering technique based on the T5 architecture.
• Score: 2.1656586298989793
• License:
• Abstract: Automatic keyphrase labelling stands for the ability of models to retrieve words or short phrases that adequately describe documents' content. Previous work has put much effort into exploring extractive techniques to address this task; however, these methods cannot produce keyphrases not found in the text. Given this limitation, keyphrase generation approaches have arisen lately. This paper presents a keyphrase generation model based on the Text-to-Text Transfer Transformer (T5) architecture. Having a document's title and abstract as input, we learn a T5 model to generate keyphrases which adequately define its content. We name this model docT5keywords. We not only perform the classic inference approach, where the output sequence is directly selected as the predicted values, but we also report results from a majority voting approach. In this approach, multiple sequences are generated, and the keyphrases are ranked based on their frequency of occurrence across these sequences. Along with this model, we present a novel keyphrase filtering technique based on the T5 architecture. We train a T5 model to learn whether a given keyphrase is relevant to a document. We devise two evaluation methodologies to prove our model's capability to filter inadequate keyphrases. First, we perform a binary evaluation where our model has to predict if a keyphrase is relevant for a given document. Second, we filter the predicted keyphrases by several AKG models and check if the evaluation scores are improved. Experimental results demonstrate that our keyphrase generation model significantly outperforms all the baselines, with gains exceeding 100\% in some cases. The proposed filtering technique also achieves near-perfect accuracy in eliminating false positives across all datasets.

Related papers

• MetaKP: On-Demand Keyphrase Generation [52.48698290354449]
  We introduce on-demand keyphrase generation, a novel paradigm that requires keyphrases that conform to specific high-level goals or intents. We present MetaKP, a large-scale benchmark comprising four datasets, 7500 documents, and 3760 goals across news and biomedical domains with human-annotated keyphrases. We demonstrate the potential of our method to serve as a general NLP infrastructure, exemplified by its application in epidemic event detection from social media.
  arXiv  Detail & Related papers  (2024-06-28T19:02:59Z)
• SimCKP: Simple Contrastive Learning of Keyphrase Representations [36.88517357720033]
  We propose SimCKP, a simple contrastive learning framework that consists of two stages: 1) An extractor-generator that extracts keyphrases by learning context-aware phrase-level representations in a contrastive manner while also generating keyphrases that do not appear in the document; and 2) A reranker that adapts scores for each generated phrase by likewise aligning their representations with the corresponding document.
  arXiv  Detail & Related papers  (2023-10-12T11:11:54Z)
• Neural Keyphrase Generation: Analysis and Evaluation [47.004575377472285]
  We study various tendencies exhibited by three strong models: T5 (based on a pre-trained transformer), CatSeq-Transformer (a non-pretrained Transformer), and ExHiRD (based on a recurrent neural network) We propose a novel metric framework, SoftKeyScore, to evaluate the similarity between two sets of keyphrases.
  arXiv  Detail & Related papers  (2023-04-27T00:10:21Z)
• Applying Transformer-based Text Summarization for Keyphrase Generation [2.28438857884398]
  Keyphrases are crucial for searching and systematizing scholarly documents. In this paper, we experiment with popular transformer-based models for abstractive text summarization. We show that summarization models are quite effective in generating keyphrases in the terms of the full-match F1-score and BERT.Score. We also investigate several ordering strategies to target keyphrases.
  arXiv  Detail & Related papers  (2022-09-08T13:01:52Z)
• Deep Keyphrase Completion [59.0413813332449]
  Keyphrase provides accurate information of document content that is highly compact, concise, full of meanings, and widely used for discourse comprehension, organization, and text retrieval. We propose textitkeyphrase completion (KPC) to generate more keyphrases for document (e.g. scientific publication) taking advantage of document content along with a very limited number of known keyphrases. We name it textitdeep keyphrase completion (DKPC) since it attempts to capture the deep semantic meaning of the document content together with known keyphrases via a deep learning framework
  arXiv  Detail & Related papers  (2021-10-29T07:15:35Z)
• Unsupervised Deep Keyphrase Generation [14.544869226959612]
  Keyphrase generation aims to summarize long documents with a collection of salient phrases. Deep neural models have demonstrated a remarkable success in this task, capable of predicting keyphrases that are even absent from a document. We present a novel method for keyphrase generation, AutoKeyGen, without the supervision of any human annotation.
  arXiv  Detail & Related papers  (2021-04-18T05:53:19Z)
• MASKER: Masked Keyword Regularization for Reliable Text Classification [73.90326322794803]
  We propose a fine-tuning method, coined masked keyword regularization (MASKER), that facilitates context-based prediction. MASKER regularizes the model to reconstruct the keywords from the rest of the words and make low-confidence predictions without enough context. We demonstrate that MASKER improves OOD detection and cross-domain generalization without degrading classification accuracy.
  arXiv  Detail & Related papers  (2020-12-17T04:54:16Z)
• Select, Extract and Generate: Neural Keyphrase Generation with Layer-wise Coverage Attention [75.44523978180317]
  We propose emphSEG-Net, a neural keyphrase generation model that is composed of two major components. The experimental results on seven keyphrase generation benchmarks from scientific and web documents demonstrate that SEG-Net outperforms the state-of-the-art neural generative methods by a large margin.
  arXiv  Detail & Related papers  (2020-08-04T18:00:07Z)
• Exclusive Hierarchical Decoding for Deep Keyphrase Generation [63.357895318562214]
  Keyphrase generation (KG) aims to summarize the main ideas of a document into a set of keyphrases. Previous work in this setting employs a sequential decoding process to generate keyphrases. We propose an exclusive hierarchical decoding framework that includes a hierarchical decoding process and either a soft or a hard exclusion mechanism.
  arXiv  Detail & Related papers  (2020-04-18T02:58:00Z)
• Keyphrase Extraction with Span-based Feature Representations [13.790461555410747]
  Keyphrases are capable of providing semantic metadata characterizing documents. Three approaches to address keyphrase extraction: (i) traditional two-step ranking method, (ii) sequence labeling and (iii) generation using neural networks. In this paper, we propose a novelty Span Keyphrase Extraction model that extracts span-based feature representation of keyphrase directly from all the content tokens.
  arXiv  Detail & Related papers  (2020-02-13T09:48:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.