Spurious Prompts: Can Irrelevant Prompts Steer Large Language Models?

• URL: http://arxiv.org/abs/2605.29678v1
• Date: Thu, 28 May 2026 09:39:44 GMT
• Title: Spurious Prompts: Can Irrelevant Prompts Steer Large Language Models?
• Abstract summary: Large language models are highly sensitive to prompts.<n>This sensitivity is usually studied through task-relevant instructions, demonstrations, or reasoning cues.<n>We call spurious prompts and show their surprising efficacy.
• Score: 17.170800056387044
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models are highly sensitive to prompts, but this sensitivity is usually studied through task-relevant instructions, demonstrations, or reasoning cues. In this paper, we study a different form of prompt sensitivity: whether prompts that are semantically unrelated to the task can nevertheless steer model behavior. We call them spurious prompts and show their surprising efficacy. We also propose a simple black-box search procedure for discovering them. Across reasoning and question-answering benchmarks, using models ranging from 0.8B to 27B parameters and spanning three model families, we show that spurious prompts can improve performance, often matching or outperforming standard prompting baselines and task-aware prompt optimization. We further show that they can steer models toward unintended behaviors, such as repeatedly selecting the first answer option, producing incorrect answers, returning an even, prime or small number without explicitly instructing the model to do so. These findings reveal a new kind of prompt sensitivity: LLMs can be systematically steered by prompts that are unrelated to the task they are asked to solve. Our code is available at https://github.com/Batorskq/spurious

Related papers

• TAPR: Enhancing LLM Performance with a Task-Aware Prompt Rewriter [1.6003479913250962]
  Large Language Models (LLMs) often require carefully crafted prompts to unlock their full potential.<n>This work addresses the challenge by introducing a Task-Aware Prompt Rewriter (TAPR), a model that reformulates user prompts into task-optimized prompts.
  arXiv  Detail & Related papers  (2026-07-17T15:37:07Z)
• Revisiting Prompt Sensitivity in Large Language Models for Text Classification: The Role of Prompt Underspecification [3.2059646106414967]
  Large language models (LLMs) are widely used as zero-shot and few-shot classifiers.<n>We study and compare the sensitivity of underspecified prompts and prompts that provide specific instructions.<n>We find that underspecified prompts exhibit higher performance variance and lower logit values for relevant tokens, while instruction-prompts suffer less from such problems.
  arXiv  Detail & Related papers  (2026-02-04T07:59:28Z)
• Promptriever: Instruction-Trained Retrievers Can Be Prompted Like Language Models [54.272894325370956]
  We present Promptriever, the first retrieval model able to be prompted like an LM. Promptriever achieves strong performance on standard retrieval tasks, and also follows instructions.
  arXiv  Detail & Related papers  (2024-09-17T12:42:55Z)
• Eliciting Human Preferences with Language Models [56.68637202313052]
  Language models (LMs) can be directed to perform target tasks by using labeled examples or natural language prompts. We propose to use *LMs themselves* to guide the task specification process. We study GATE in three domains: email validation, content recommendation, and moral reasoning.
  arXiv  Detail & Related papers  (2023-10-17T21:11:21Z)
• Demystifying Prompts in Language Models via Perplexity Estimation [109.59105230163041]
  Performance of a prompt is coupled with the extent to which the model is familiar with the language it contains. We show that the lower the perplexity of the prompt is, the better the prompt is able to perform the task.
  arXiv  Detail & Related papers  (2022-12-08T02:21:47Z)
• Ask Me Anything: A simple strategy for prompting language models [24.294416731247427]
  Large language models (LLMs) transfer well to new tasks out-of-the-box simply given a natural language prompt. We develop an understanding of the effective prompt formats, finding that question-answering (QA) prompts tend to outperform those that restrict the model outputs. We apply the collected prompts to obtain several noisy votes for the input's true label. We find that the prompts can have very different accuracies and complex dependencies.
  arXiv  Detail & Related papers  (2022-10-05T17:59:45Z)
• Bayesian Prompt Learning for Image-Language Model Generalization [64.50204877434878]
  We use the regularization ability of Bayesian methods to frame prompt learning as a variational inference problem. Our approach regularizes the prompt space, reduces overfitting to the seen prompts and improves the prompt generalization on unseen prompts. We demonstrate empirically on 15 benchmarks that Bayesian prompt learning provides an appropriate coverage of the prompt space.
  arXiv  Detail & Related papers  (2022-10-05T17:05:56Z)
• Interactive and Visual Prompt Engineering for Ad-hoc Task Adaptation with Large Language Models [116.25562358482962]
  State-of-the-art neural language models can be used to solve ad-hoc language tasks without the need for supervised training. PromptIDE allows users to experiment with prompt variations, visualize prompt performance, and iteratively optimize prompts.
  arXiv  Detail & Related papers  (2022-08-16T17:17:53Z)
• Least-to-Most Prompting Enables Complex Reasoning in Large Language Models [52.59923418570378]
  We propose a novel prompting strategy, least-to-most prompting, to overcome the challenge of easy-to-hard generalization. We show that least-to-most prompting is capable of generalizing to more difficult problems than those seen in prompts. neural-symbolic models in the literature that specialize in solving SCAN are trained on the entire training set containing over 15,000 examples.
  arXiv  Detail & Related papers  (2022-05-21T15:34:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.