Select, Extract and Generate: Neural Keyphrase Generation with Layer-wise Coverage Attention

• URL: http://arxiv.org/abs/2008.01739v2
• Date: Fri, 4 Jun 2021 20:50:29 GMT
• Title: Select, Extract and Generate: Neural Keyphrase Generation with Layer-wise Coverage Attention
• Authors: Wasi Uddin Ahmad and Xiao Bai and Soomin Lee and Kai-Wei Chang
• Abstract summary: We propose emphSEG-Net, a neural keyphrase generation model that is composed of two major components. The experimental results on seven keyphrase generation benchmarks from scientific and web documents demonstrate that SEG-Net outperforms the state-of-the-art neural generative methods by a large margin.
• Score: 75.44523978180317
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Natural language processing techniques have demonstrated promising results in keyphrase generation. However, one of the major challenges in \emph{neural} keyphrase generation is processing long documents using deep neural networks. Generally, documents are truncated before given as inputs to neural networks. Consequently, the models may miss essential points conveyed in the target document. To overcome this limitation, we propose \emph{SEG-Net}, a neural keyphrase generation model that is composed of two major components, (1) a selector that selects the salient sentences in a document and (2) an extractor-generator that jointly extracts and generates keyphrases from the selected sentences. SEG-Net uses Transformer, a self-attentive architecture, as the basic building block with a novel \emph{layer-wise} coverage attention to summarize most of the points discussed in the document. The experimental results on seven keyphrase generation benchmarks from scientific and web documents demonstrate that SEG-Net outperforms the state-of-the-art neural generative methods by a large margin.

Related papers

• Retrieval is Accurate Generation [99.24267226311157]
  We introduce a novel method that selects context-aware phrases from a collection of supporting documents. Our model achieves the best performance and the lowest latency among several retrieval-augmented baselines.
  arXiv  Detail & Related papers  (2024-02-27T14:16:19Z)
• SimCKP: Simple Contrastive Learning of Keyphrase Representations [36.88517357720033]
  We propose SimCKP, a simple contrastive learning framework that consists of two stages: 1) An extractor-generator that extracts keyphrases by learning context-aware phrase-level representations in a contrastive manner while also generating keyphrases that do not appear in the document; and 2) A reranker that adapts scores for each generated phrase by likewise aligning their representations with the corresponding document.
  arXiv  Detail & Related papers  (2023-10-12T11:11:54Z)
• Neural Keyphrase Generation: Analysis and Evaluation [47.004575377472285]
  We study various tendencies exhibited by three strong models: T5 (based on a pre-trained transformer), CatSeq-Transformer (a non-pretrained Transformer), and ExHiRD (based on a recurrent neural network) We propose a novel metric framework, SoftKeyScore, to evaluate the similarity between two sets of keyphrases.
  arXiv  Detail & Related papers  (2023-04-27T00:10:21Z)
• HETFORMER: Heterogeneous Transformer with Sparse Attention for Long-Text Extractive Summarization [57.798070356553936]
  HETFORMER is a Transformer-based pre-trained model with multi-granularity sparse attentions for extractive summarization. Experiments on both single- and multi-document summarization tasks show that HETFORMER achieves state-of-the-art performance in Rouge F1.
  arXiv  Detail & Related papers  (2021-10-12T22:42:31Z)
• SGG: Learning to Select, Guide, and Generate for Keyphrase Generation [38.351526320316786]
  Keyphrases concisely summarize the high-level topics discussed in a document. Most existing keyphrase generation approaches synchronously generate present and absent keyphrases. We propose a Select-Guide-Generate (SGG) approach to deal with present and absent keyphrase generation separately.
  arXiv  Detail & Related papers  (2021-05-06T09:43:33Z)
• Be More with Less: Hypergraph Attention Networks for Inductive Text Classification [56.98218530073927]
  Graph neural networks (GNNs) have received increasing attention in the research community and demonstrated their promising results on this canonical task. Despite the success, their performance could be largely jeopardized in practice since they are unable to capture high-order interaction between words. We propose a principled model -- hypergraph attention networks (HyperGAT) which can obtain more expressive power with less computational consumption for text representation learning.
  arXiv  Detail & Related papers  (2020-11-01T00:21:59Z)
• Keyphrase Extraction with Dynamic Graph Convolutional Networks and Diversified Inference [50.768682650658384]
  Keyphrase extraction (KE) aims to summarize a set of phrases that accurately express a concept or a topic covered in a given document. Recent Sequence-to-Sequence (Seq2Seq) based generative framework is widely used in KE task, and it has obtained competitive performance on various benchmarks. In this paper, we propose to adopt the Dynamic Graph Convolutional Networks (DGCN) to solve the above two problems simultaneously.
  arXiv  Detail & Related papers  (2020-10-24T08:11:23Z)
• A Joint Learning Approach based on Self-Distillation for Keyphrase Extraction from Scientific Documents [29.479331909227998]
  Keyphrase extraction is the task of extracting a small set of phrases that best describe a document. Most existing benchmark datasets for the task typically have limited numbers of annotated documents. We propose a simple and efficient joint learning approach based on the idea of self-distillation.
  arXiv  Detail & Related papers  (2020-10-22T18:36:31Z)
• Keyphrase Generation with Cross-Document Attention [28.565813544820553]
  Keyphrase generation aims to produce a set of phrases summarizing the essentials of a given document. We propose CDKGen, a Transformer-based keyphrase generator, which expands the Transformer to global attention. We also adopt a copy mechanism to enhance our model via selecting appropriate words from documents to deal with out-of-vocabulary words in keyphrases.
  arXiv  Detail & Related papers  (2020-04-21T07:58:27Z)
• Keyphrase Extraction with Span-based Feature Representations [13.790461555410747]
  Keyphrases are capable of providing semantic metadata characterizing documents. Three approaches to address keyphrase extraction: (i) traditional two-step ranking method, (ii) sequence labeling and (iii) generation using neural networks. In this paper, we propose a novelty Span Keyphrase Extraction model that extracts span-based feature representation of keyphrase directly from all the content tokens.
  arXiv  Detail & Related papers  (2020-02-13T09:48:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.