A Survey on Prompting Techniques in LLMs

• URL: http://arxiv.org/abs/2312.03740v2
• Date: Tue, 16 Apr 2024 22:27:39 GMT
• Title: A Survey on Prompting Techniques in LLMs
• Authors: Prabin Bhandari,
• Abstract summary: Autoregressive Large Language Models have transformed the landscape of Natural Language Processing. We present a taxonomy of existing literature on prompting techniques and provide a concise survey based on this taxonomy. We identify some open problems in the realm of prompting in autoregressive LLMs which could serve as a direction for future research.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Autoregressive Large Language Models have transformed the landscape of Natural Language Processing. Pre-train and prompt paradigm has replaced the conventional approach of pre-training and fine-tuning for many downstream NLP tasks. This shift has been possible largely due to LLMs and innovative prompting techniques. LLMs have shown great promise for a variety of downstream tasks owing to their vast parameters and huge datasets that they are pre-trained on. However, in order to fully realize their potential, their outputs must be guided towards the desired outcomes. Prompting, in which a specific input or instruction is provided to guide the LLMs toward the intended output, has become a tool for achieving this goal. In this paper, we discuss the various prompting techniques that have been applied to fully harness the power of LLMs. We present a taxonomy of existing literature on prompting techniques and provide a concise survey based on this taxonomy. Further, we identify some open problems in the realm of prompting in autoregressive LLMs which could serve as a direction for future research.

Related papers

• A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Tasks [0.0]
  Large language models (LLMs) have shown remarkable performance on many different Natural Language Processing (NLP) tasks. Prompt engineering plays a key role in adding more to the already existing abilities of LLMs to achieve significant performance gains. This paper summarizes different prompting techniques and club them together based on different NLP tasks that they have been used for.
  arXiv  Detail & Related papers  (2024-07-17T20:23:19Z)
• MAPO: Boosting Large Language Model Performance with Model-Adaptive Prompt Optimization [73.7779735046424]
  We show that different prompts should be adapted to different Large Language Models (LLM) to enhance their capabilities across various downstream tasks in NLP. We then propose a model-adaptive prompt (MAPO) method that optimize the original prompts for each specific LLM in downstream tasks.
  arXiv  Detail & Related papers  (2024-07-04T18:39:59Z)
• RePrompt: Planning by Automatic Prompt Engineering for Large Language Models Agents [27.807695570974644]
  Large language models (LLMs) have had remarkable success in domains outside the traditional natural language processing. We propose a novel method, textscRePrompt, which does "gradient descent" to optimize the step-by-step instructions in the prompt of the LLM agents.
  arXiv  Detail & Related papers  (2024-06-17T01:23:11Z)
• Large Language Models Meet NLP: A Survey [79.74450825763851]
  Large language models (LLMs) have shown impressive capabilities in Natural Language Processing (NLP) tasks. This study aims to address this gap by exploring the following questions.
  arXiv  Detail & Related papers  (2024-05-21T14:24:01Z)
• Efficient Prompting Methods for Large Language Models: A Survey [50.171011917404485]
  Prompting has become a mainstream paradigm for adapting large language models (LLMs) to specific natural language processing tasks. This approach brings the additional computational burden of model inference and human effort to guide and control the behavior of LLMs. We present the basic concepts of prompting, review the advances for efficient prompting, and highlight future research directions.
  arXiv  Detail & Related papers  (2024-04-01T12:19:08Z)
• Using Large Language Models for Natural Language Processing Tasks in Requirements Engineering: A Systematic Guideline [2.6644624823848426]
  Large Language Models (LLMs) are the cornerstone in automating Requirements Engineering (RE) tasks. This chapter aims to furnish readers with essential knowledge about LLMs in its initial segment. It provides a comprehensive guideline tailored for students, researchers, and practitioners on harnessing LLMs to address their specific objectives.
  arXiv  Detail & Related papers  (2024-02-21T14:00:52Z)
• Large Language Models: A Survey [69.72787936480394]
  Large Language Models (LLMs) have drawn a lot of attention due to their strong performance on a wide range of natural language tasks. LLMs' ability of general-purpose language understanding and generation is acquired by training billions of model's parameters on massive amounts of text data.
  arXiv  Detail & Related papers  (2024-02-09T05:37:09Z)
• Continual Learning for Large Language Models: A Survey [95.79977915131145]
  Large language models (LLMs) are not amenable to frequent re-training, due to high training costs arising from their massive scale. This paper surveys recent works on continual learning for LLMs.
  arXiv  Detail & Related papers  (2024-02-02T12:34:09Z)
• Prompts Matter: Insights and Strategies for Prompt Engineering in Automated Software Traceability [45.235173351109374]
  Large Language Models (LLMs) have the potential to revolutionize automated traceability. This paper explores the process of prompt engineering to extract link predictions from an LLM.
  arXiv  Detail & Related papers  (2023-08-01T01:56:22Z)
• Aligning Large Language Models with Human: A Survey [53.6014921995006]
  Large Language Models (LLMs) trained on extensive textual corpora have emerged as leading solutions for a broad array of Natural Language Processing (NLP) tasks. Despite their notable performance, these models are prone to certain limitations such as misunderstanding human instructions, generating potentially biased content, or factually incorrect information. This survey presents a comprehensive overview of these alignment technologies, including the following aspects.
  arXiv  Detail & Related papers  (2023-07-24T17:44:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.