You Shouldn't Have Asked: A Pragmatics-Inspired Taxonomy for Evaluating LLM Refusals

• URL: http://arxiv.org/abs/2608.30856v1
• Date: Mon, 31 Aug 2026 14:19:16 GMT
• Title: You Shouldn't Have Asked: A Pragmatics-Inspired Taxonomy for Evaluating LLM Refusals
• Abstract summary: We propose the first taxonomy of LLM refusals grounded in pragmatic theory.<n>Although models differ in how they refuse, their refusals are overall explicit and strongly morally evaluative.<n>We call for alignment evaluation that considers not only whether models refuse harmful requests, but also whether they refuse in ways that are contextually adaptive and socially accountable for the interactional consequences of saying no.
• Score: 6.358967433398892
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Refusals are often treated as face-threatening acts in pragmatics because they can challenge the requester's socially claimed self-image. Large language models (LLMs) are increasingly trained to refuse unsafe and inappropriate requests, and these refusals may harm users when models fail to manage this interactional cost properly. While existing work has mainly approached LLM non-compliance as a safety-alignment outcome, it does not provide a way to evaluate whether LLMs refuse appropriately across different harmful contexts. To study this question, we propose (to our knowledge) the first taxonomy of LLM refusals that is grounded in pragmatic theory. Applying this taxonomy to responses from 16 modern LLMs across 14 harm categories, we find that although models differ in how they refuse, their refusals are overall explicit and strongly morally evaluative, with interactional repair occurring mainly through offering or providing safer alternatives instead of interpersonal facework. This pattern is especially consequential in sensitive harm contexts, where overuse of negative framing may make users feel shamed or provoked, undermining the purpose of safe non-compliance. We therefore call for alignment evaluation that considers not only whether models refuse harmful requests, but also whether they refuse in ways that are contextually adaptive and socially accountable for the interactional consequences of saying no.

Related papers

• LivingArena: Do LLMs Know What Other LLMs Don't? Peer-Probing as Scalable Evaluation [79.03892269184145]
  LivingArena is an automated, contamination-resistant evaluation framework.<n>Models take turns proposing questions, aiming to pose items that opponents cannot answer correctly.<n>Questioners are encouraged to actively identify and exploit opponents' knowledge boundaries.
  arXiv  Detail & Related papers  (2026-06-19T06:20:58Z)
• The Ethics of LLM Sandbox and Persona Dynamics [0.0]
  We argue that actively generating reality gaps is unethical because it knowingly shifts risk back to the uninformed user.<n>The risk is sharpest in high-exposure advice contexts, where users seek orientation rather than a bounded, externally checkable task.
  arXiv  Detail & Related papers  (2026-05-27T15:52:07Z)
• Beyond "I cannot fulfill this request": Alleviating Rigid Rejection in LLMs via Label Enhancement [39.705063586885075]
  Large Language Models (LLMs) rely on safety alignment to obey safe requests while refusing harmful ones.<n>This paper proposes LANCE to ensure safe yet flexible and natural responses via label enhancement.<n> Experiments demonstrate that LANCE significantly alleviates the rigid rejection problem while maintaining high security standards.
  arXiv  Detail & Related papers  (2026-05-08T15:33:54Z)
• Verbalizing LLMs' assumptions to explain and control sycophancy [62.927670321859495]
  LLMs can be socially sycophantic, affirming users when they ask questions like "am I in the wrong?" rather than providing genuine assessment.<n>We present Verbalized Assumptions, a framework for eliciting these assumptions from LLMs.<n>Our work contributes a new understanding of assumptions as a mechanism for sycophancy.
  arXiv  Detail & Related papers  (2026-04-03T14:15:43Z)
• Deactivating Refusal Triggers: Understanding and Mitigating Overrefusal in Safety Alignment [14.672527265339722]
  Safety alignment aims to ensure that large language models (LLMs) refuse harmful requests by post-training on harmful queries paired with refusal answers.<n>In this paper, we examine how overrefusal arises under safety alignment, and propose a mitigation strategy inspired by our findings.
  arXiv  Detail & Related papers  (2026-03-12T00:07:19Z)
• Learning to Extract Context for Context-Aware LLM Inference [60.376872353918394]
  User prompts to large language models (LLMs) are often ambiguous or under-specified.<n> contextual cues shaped by user intentions, prior knowledge, and risk factors influence what constitutes an appropriate response.<n>We propose a framework that extracts and leverages such contextual information from the user prompt itself.
  arXiv  Detail & Related papers  (2025-12-12T19:10:08Z)
• Think Before Refusal : Triggering Safety Reflection in LLMs to Mitigate False Refusal Behavior [59.20260988638777]
  We demonstrate that prompting safety reflection before generating a response can mitigate false refusal behavior.<n>In an ablation study across 15 pre-trained models, we show that models fine-tuned with safety reflection significantly reduce false refusal behavior.
  arXiv  Detail & Related papers  (2025-03-22T23:35:49Z)
• LLM Content Moderation and User Satisfaction: Evidence from Response Refusals in Chatbot Arena [0.0]
  We show that ethical refusals yield significantly lower win rates than both technical refusals and standard responses.<n>Our findings underscore a core tension in LLM design: safety-aligned behaviors may conflict with user expectations.
  arXiv  Detail & Related papers  (2025-01-04T06:36:44Z)
• Refuse Whenever You Feel Unsafe: Improving Safety in LLMs via Decoupled Refusal Training [67.30423823744506]
  We introduce a novel approach, Decoupled Refusal Training (DeRTa), designed to empower LLMs to refuse compliance to harmful prompts at any response position.<n>DeRTa incorporates two novel components: (1) Maximum Likelihood Estimation with Harmful Response Prefix, which trains models to recognize and avoid unsafe content by appending a segment of harmful response to the beginning of a safe response, and (2) Reinforced Transition Optimization (RTO), which equips models with the ability to transition from potential harm to safety refusal consistently throughout the harmful response sequence.
  arXiv  Detail & Related papers  (2024-07-12T09:36:33Z)
• On Prompt-Driven Safeguarding for Large Language Models [172.13943777203377]
  We find that in the representation space, the input queries are typically moved by safety prompts in a "higher-refusal" direction. Inspired by these findings, we propose a method for safety prompt optimization, namely DRO. Treating a safety prompt as continuous, trainable embeddings, DRO learns to move the queries' representations along or opposite the refusal direction, depending on their harmfulness.
  arXiv  Detail & Related papers  (2024-01-31T17:28:24Z)
• Examining LLMs' Uncertainty Expression Towards Questions Outside Parametric Knowledge [35.067234242461545]
  Large language models (LLMs) express uncertainty in situations where they lack sufficient parametric knowledge to generate reasonable responses. This work aims to systematically investigate LLMs' behaviors in such situations, emphasizing the trade-off between honesty and helpfulness.
  arXiv  Detail & Related papers  (2023-11-16T10:02:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.