Do Prompt-Based Models Really Understand the Meaning of their Prompts?

• URL: http://arxiv.org/abs/2109.01247v1
• Date: Thu, 2 Sep 2021 23:46:36 GMT
• Title: Do Prompt-Based Models Really Understand the Meaning of their Prompts?
• Authors: Albert Webson, Ellie Pavlick
• Abstract summary: We find that models learn just as fast with many prompts that are intentionally irrelevant or even pathologically misleading. We find little evidence that suggests existing prompt-based models truly understand the meaning of their given prompts.
• Score: 12.857580576554865
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recently, a boom of papers have shown extraordinary progress in few-shot learning with various prompt-based models. Such success can give the impression that prompts help models to learn faster in the same way that humans learn faster when provided with task instructions expressed in natural language. In this study, we experiment with over 30 prompts manually written for natural language inference (NLI). We find that models learn just as fast with many prompts that are intentionally irrelevant or even pathologically misleading as they do with instructively "good" prompts. Additionally, we find that model performance is more dependent on the choice of the LM target words (a.k.a. the "verbalizer" that converts LM vocabulary prediction to class labels) than on the text of the prompt itself. In sum, we find little evidence that suggests existing prompt-based models truly understand the meaning of their given prompts.

Related papers

• Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Models [54.272894325370956]
  We present Promptriever, the first retrieval model able to be prompted like an LM. Promptriever achieves strong performance on standard retrieval tasks, and also follows instructions.
  arXiv  Detail & Related papers  (2024-09-17T12:42:55Z)
• An Incomplete Loop: Deductive, Inductive, and Abductive Learning in Large Language Models [99.31449616860291]
  Modern language models (LMs) can learn to perform new tasks in different ways. In instruction following, the target task is described explicitly in natural language; in few-shot prompting, the task is specified implicitly. In instruction inference, LMs are presented with in-context examples and are then prompted to generate a natural language task description.
  arXiv  Detail & Related papers  (2024-04-03T19:31:56Z)
• Effective Prompt Extraction from Language Models [70.00099540536382]
  We present a framework for measuring the effectiveness of prompt extraction attacks. In experiments with 3 different sources of prompts and 11 underlying large language models, we find that simple text-based attacks can in fact reveal prompts with high probability. Our framework determines with high precision whether an extracted prompt is the actual secret prompt, rather than a model hallucination.
  arXiv  Detail & Related papers  (2023-07-13T16:15:08Z)
• TreePrompt: Learning to Compose Tree Prompts for Explainable Visual Grounding [17.9785504685384]
  We propose a new prompt construction paradigm with explicit explainable ability, named TreePrompt. Specifically, we first deconstruct a complex sentence into a tree, that is consistent with human reasoning. Thanks to this step-by-step prompt construction process, each intermediate prompt (i.e., tree node) permits us to understand the reasoning process.
  arXiv  Detail & Related papers  (2023-05-19T07:52:22Z)
• Can discrete information extraction prompts generalize across language models? [36.85568212975316]
  We study whether automatically-induced prompts can also be used, out-of-the-box, to probe other language models for the same information. We introduce a way to induce prompts by mixing language models at training time that results in prompts that generalize well across models.
  arXiv  Detail & Related papers  (2023-02-20T09:56:51Z)
• Demystifying Prompts in Language Models via Perplexity Estimation [109.59105230163041]
  Performance of a prompt is coupled with the extent to which the model is familiar with the language it contains. We show that the lower the perplexity of the prompt is, the better the prompt is able to perform the task.
  arXiv  Detail & Related papers  (2022-12-08T02:21:47Z)
• Extensible Prompts for Language Models on Zero-shot Language Style Customization [89.1622516945109]
  X-Prompt instructs a large language model (LLM) beyond natural language (NL) registering new imaginary words allows us to instruct the LLM to comprehend concepts that are difficult to describe with NL words. These imaginary words are designed to be out-of-distribution robust so that they can be (re)used like NL words in various prompts.
  arXiv  Detail & Related papers  (2022-12-01T16:11:56Z)
• Do Prompts Solve NLP Tasks Using Natural Language? [18.611748762251494]
  In this work, we empirically compare the three types of prompts under both few-shot and fully-supervised settings. Our experimental results show that schema prompts are the most effective in general.
  arXiv  Detail & Related papers  (2022-03-02T07:20:59Z)
• Instance-aware Prompt Learning for Language Understanding and Generation [49.22899822734549]
  We propose an instance-aware prompt learning method that learns a different prompt for each instance. Our method achieves the state-of-the-art on the SuperGLUE few-shot learning benchmark.
  arXiv  Detail & Related papers  (2022-01-18T17:03:25Z)
• Learning How to Ask: Querying LMs with Mixtures of Soft Prompts [33.43689407735244]
  Natural-language prompts have recently been used to coax pretrained language models into performing other AI tasks. We explore the idea of learning prompts by gradient descent. For each task, we optimize a mixture of prompts, learning which prompts are most effective and how to ensemble them.
  arXiv  Detail & Related papers  (2021-04-14T02:56:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.