Zero-Shot Keyphrase Generation: Investigating Specialized Instructions and Multi-Sample Aggregation on Large Language Models

• URL: http://arxiv.org/abs/2503.00597v1
• Date: Sat, 01 Mar 2025 19:38:57 GMT
• Title: Zero-Shot Keyphrase Generation: Investigating Specialized Instructions and Multi-Sample Aggregation on Large Language Models
• Authors: Jayanth Mohan, Jishnu Ray Chowdhury, Tomas Malik, Cornelia Caragea,
• Abstract summary: Keyphrase generation is a long-standing NLP task for automatically generating keyphrases for a given document.<n>We focus on the zero-shot capabilities of open-source instruction-tuned LLMs (Phi-3, Llama-3) and the closed-source GPT-4o for this task.
• Score: 52.829293635314194
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Keyphrases are the essential topical phrases that summarize a document. Keyphrase generation is a long-standing NLP task for automatically generating keyphrases for a given document. While the task has been comprehensively explored in the past via various models, only a few works perform some preliminary analysis of Large Language Models (LLMs) for the task. Given the impact of LLMs in the field of NLP, it is important to conduct a more thorough examination of their potential for keyphrase generation. In this paper, we attempt to meet this demand with our research agenda. Specifically, we focus on the zero-shot capabilities of open-source instruction-tuned LLMs (Phi-3, Llama-3) and the closed-source GPT-4o for this task. We systematically investigate the effect of providing task-relevant specialized instructions in the prompt. Moreover, we design task-specific counterparts to self-consistency-style strategies for LLMs and show significant benefits from our proposals over the baselines.

Related papers

• Key Algorithms for Keyphrase Generation: Instruction-Based LLMs for Russian Scientific Keyphrases [2.8402080392117757]
  Keyphrase selection is a challenging task in natural language processing that has a wide range of applications. Adapting existing supervised and unsupervised solutions for the Russian language faces several limitations. Recent studies conducted on English texts show that large language models (LLMs) successfully address the task of generating keyphrases.
  arXiv  Detail & Related papers  (2024-10-23T17:07:32Z)
• CriSPO: Multi-Aspect Critique-Suggestion-guided Automatic Prompt Optimization for Text Generation [18.39379838806384]
  We propose a novel critique-suggestion-guided automatic Prompt Optimization (CriSPO) approach.<n>CriSPO introduces a critique-suggestion module as its core component.<n>This module spontaneously discovers aspects, and compares generated reference texts across these aspects, providing actionable suggestions for prompt modification.<n>To further improve CriSPO with multi-metric optimization, we introduce an Automatic Suffix Tuning (AST) extension to enhance the performance of task prompts across multiple metrics.
  arXiv  Detail & Related papers  (2024-10-03T17:57:01Z)
• Salient Information Prompting to Steer Content in Prompt-based Abstractive Summarization [4.9201947803787744]
  Large language models (LLMs) can generate fluent summaries across domains using prompting techniques.<n>We show that adding keyphrases in prompts can improve ROUGE F1 and recall.<n>We introduce Keyphrase Signal Extractor (SigExt), a lightweight model that can be finetuned to extract salient keyphrases.
  arXiv  Detail & Related papers  (2024-10-03T17:54:56Z)
• ZeroDL: Zero-shot Distribution Learning for Text Clustering via Large Language Models [5.011816280731356]
  We propose a simple yet effective method to contextualize a task toward a specific large language model (LLMs) We show the effectiveness of this approach in text clustering tasks, and also highlight the importance of the contextualization through examples of the above procedure.
  arXiv  Detail & Related papers  (2024-06-19T08:48:05Z)
• Pushing the Limits of ChatGPT on NLP Tasks [79.17291002710517]
  Despite the success of ChatGPT, its performances on most NLP tasks are still well below the supervised baselines. In this work, we looked into the causes, and discovered that its subpar performance was caused by the following factors. We propose a collection of general modules to address these issues, in an attempt to push the limits of ChatGPT on NLP tasks.
  arXiv  Detail & Related papers  (2023-06-16T09:40:05Z)
• PEARL: Prompting Large Language Models to Plan and Execute Actions Over Long Documents [78.27865456183397]
  We propose PEARL, a prompting framework to improve reasoning over long documents. Each stage of PEARL is implemented via zero-shot or few-shot prompting with minimal human input. We evaluate PEARL on a challenging subset of the QuALITY dataset, which contains questions that require complex reasoning over long narrative texts.
  arXiv  Detail & Related papers  (2023-05-23T23:06:04Z)
• Multi-Task Instruction Tuning of LLaMa for Specific Scenarios: A Preliminary Study on Writing Assistance [60.40541387785977]
  Small foundational models can display remarkable proficiency in tackling diverse tasks when fine-tuned using instruction-driven data. In this work, we investigate a practical problem setting where the primary focus is on one or a few particular tasks rather than general-purpose instruction following. Experimental results show that fine-tuning LLaMA on writing instruction data significantly improves its ability on writing tasks.
  arXiv  Detail & Related papers  (2023-05-22T16:56:44Z)
• AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotators [98.11286353828525]
  GPT-3.5 series models have demonstrated remarkable few-shot and zero-shot ability across various NLP tasks. We propose AnnoLLM, which adopts a two-step approach, explain-then-annotate. We build the first conversation-based information retrieval dataset employing AnnoLLM.
  arXiv  Detail & Related papers  (2023-03-29T17:03:21Z)
• Pre-training via Paraphrasing [96.79972492585112]
  We introduce MARGE, a pre-trained sequence-to-sequence model learned with an unsupervised multi-lingual paraphrasing objective. We show it is possible to jointly learn to do retrieval and reconstruction, given only a random initialization. For example, with no additional task-specific training we achieve BLEU scores of up to 35.8 for document translation.
  arXiv  Detail & Related papers  (2020-06-26T14:43:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.