No Universal Courtesy: A Cross-Linguistic, Multi-Model Study of Politeness Effects on LLMs Using the PLUM Corpus

• URL: http://arxiv.org/abs/2604.16275v1
• Date: Fri, 17 Apr 2026 17:33:48 GMT
• Title: No Universal Courtesy: A Cross-Linguistic, Multi-Model Study of Politeness Effects on LLMs Using the PLUM Corpus
• Abstract summary: The Politeness Theory by Brown and Levinson and the Impoliteness Framework by Culpeper form the basis of experiments conducted across three languages.<n>Our sample consists of 22,500 pairs of prompts and responses of various types, evaluated across five levels of politeness.<n>The findings show that model performance is highly influenced by tone, dialogue history, and language.
• Score: 1.263738813992273
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper explores the response of Large Language Models (LLMs) to user prompts with different degrees of politeness and impoliteness. The Politeness Theory by Brown and Levinson and the Impoliteness Framework by Culpeper form the basis of experiments conducted across three languages (English, Hindi, Spanish), five models (Gemini-Pro, GPT-4o Mini, Claude 3.7 Sonnet, DeepSeek-Chat, and Llama 3), and three interaction histories between users (raw, polite, and impolite). Our sample consists of 22,500 pairs of prompts and responses of various types, evaluated across five levels of politeness using an eight-factor assessment framework: coherence, clarity, depth, responsiveness, context retention, toxicity, conciseness, and readability. The findings show that model performance is highly influenced by tone, dialogue history, and language. While polite prompts enhance the average response quality by up to ~11% and impolite tones worsen it, these effects are neither consistent nor universal across languages and models. English is best served by courteous or direct tones, Hindi by deferential and indirect tones, and Spanish by assertive tones. Among the models, Llama is the most tone-sensitive (11.5% range), whereas GPT is more robust to adversarial tone. These results indicate that politeness is a quantifiable computational variable that affects LLM behaviour, though its impact is language- and model-dependent rather than universal. To support reproducibility and future work, we additionally release PLUM (Politeness Levels in Utterances, Multilingual), a publicly available corpus of 1,500 human-validated prompts across three languages and five politeness categories, and provide a formal supplementary analysis of six falsifiable hypotheses derived from politeness theory, empirically assessed against the dataset.

Related papers

• Not What, But How: A Framework for Auditing LLM Responses across Positioning, Generalization, Anthromorphism, and Maxims [54.31707479192195]
  Large language models (LLMs) are being increasingly used to answer subjective, information-seeking questions.<n>Existing evaluations for subjective cultural queries largely focus on factual correctness, ignoring how the response is framed.<n>We introduce FRANZ, an automated FRAmework for respoNse characteriZation to conduct communicative audit of LLM responses along four dimensions.
  arXiv  Detail & Related papers  (2026-06-01T17:01:31Z)
• The Rise of Verbal Tics in Large Language Models: A Systematic Analysis Across Frontier Models [12.320824168302908]
  We present a systematic analysis of the verbal tic phenomenon across eight state-of-the-art Large Language Models (LLMs)<n>We introduce the Verbal Tic Index (VTI), a composite metric quantifying tic prevalence, and analyze its correlation with sycophancy, lexical diversity, and human-perceived naturalness.
  arXiv  Detail & Related papers  (2026-04-21T06:43:01Z)
• Investigating the Influence of Language on Sycophantic Behavior of Multilingual LLMs [1.7778609937758327]
  Large language models (LLMs) have achieved strong performance across a wide range of tasks, but they are also prone to sycophancy.<n>Previous research has outlined both the extent and the underlying causes of sycophancy in earlier models, such as ChatGPT-3.5 and Davinci.<n>This work investigates how the language influences sycophantic responses.
  arXiv  Detail & Related papers  (2026-03-29T12:31:05Z)
• On the Fallacy of Global Token Perplexity in Spoken Language Model Evaluation [88.77441715819366]
  Generative spoken language models pretrained on large-scale raw audio can continue a speech prompt with appropriate content.<n>We propose a variety of likelihood- and generative-based evaluation methods that serve in place of naive global token perplexity.
  arXiv  Detail & Related papers  (2026-01-09T22:01:56Z)
• Evaluating Modern Large Language Models on Low-Resource and Morphologically Rich Languages:A Cross-Lingual Benchmark Across Cantonese, Japanese, and Turkish [12.286855282078305]
  GPT-4o, GPT-4, Claude3.5Sonnet, LLaMA3.1, MistralLarge2, LLaMA-2Chat13B, and Mistral7BInstruct are evaluated.<n>Our benchmark spans four diverse tasks: open-domain question answering, document summarization, English-to-X translation, and culturally grounded dialogue.
  arXiv  Detail & Related papers  (2025-11-05T22:09:53Z)
• Mind Your Tone: Investigating How Prompt Politeness Affects LLM Accuracy (short paper) [0.0]
  This study investigates how varying levels of prompt politeness affect model accuracy on multiple-choice questions.<n>We created a dataset of 50 base questions spanning mathematics, science, and history, each rewritten into five tone variants: Very Polite, Polite, Neutral, Rude, and Very Rude.<n> Contrary to expectations, impolite prompts consistently outperformed polite ones, with accuracy ranging from 80.8% for Very Polite prompts to 84.8% for Very Rude prompts.
  arXiv  Detail & Related papers  (2025-10-06T15:50:39Z)
• CAPE: Context-Aware Personality Evaluation Framework for Large Language Models [8.618075786777219]
  We propose the first Context-Aware Personality Evaluation framework for Large Language Models (LLMs)<n>Our experiments reveal that conversational history enhances response consistency via in-context learning but also induces personality shifts.<n>Our framework can be applied to Role Playing Agents (RPAs) to better align with human judgments.
  arXiv  Detail & Related papers  (2025-08-28T03:17:47Z)
• Large Language Models Reflect the Ideology of their Creators [71.65505524599888]
  Large language models (LLMs) are trained on vast amounts of data to generate natural language.<n>This paper shows that the ideological stance of an LLM appears to reflect the worldview of its creators.
  arXiv  Detail & Related papers  (2024-10-24T04:02:30Z)
• Rel-A.I.: An Interaction-Centered Approach To Measuring Human-LM Reliance [73.19687314438133]
  We study how reliance is affected by contextual features of an interaction. We find that contextual characteristics significantly affect human reliance behavior. Our results show that calibration and language quality alone are insufficient in evaluating the risks of human-LM interactions.
  arXiv  Detail & Related papers  (2024-07-10T18:00:05Z)
• Paralinguistics-Enhanced Large Language Modeling of Spoken Dialogue [71.15186328127409]
  Paralinguistics-enhanced Generative Pretrained Transformer (ParalinGPT) Model takes the conversational context of text, speech embeddings, and paralinguistic attributes as input prompts within a serialized multitasking framework. We utilize the Switchboard-1 corpus, including its sentiment labels as the paralinguistic attribute, as our spoken dialogue dataset.
  arXiv  Detail & Related papers  (2023-12-23T18:14:56Z)
• Let Models Speak Ciphers: Multiagent Debate through Embeddings [84.20336971784495]
  We introduce CIPHER (Communicative Inter-Model Protocol Through Embedding Representation) to address this issue. By deviating from natural language, CIPHER offers an advantage of encoding a broader spectrum of information without any modification to the model weights. This showcases the superiority and robustness of embeddings as an alternative "language" for communication among LLMs.
  arXiv  Detail & Related papers  (2023-10-10T03:06:38Z)
• TyDiP: A Dataset for Politeness Classification in Nine Typologically Diverse Languages [33.540256516320326]
  We study politeness phenomena in nine typologically diverse languages. We create TyDiP, a dataset containing three-way politeness annotations for 500 examples in each language.
  arXiv  Detail & Related papers  (2022-11-29T18:58:15Z)
• On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatment [59.995385574274785]
  We show that, contrary to previous belief, negative interference also impacts low-resource languages. We present a meta-learning algorithm that obtains better cross-lingual transferability and alleviates negative interference.
  arXiv  Detail & Related papers  (2020-10-06T20:48:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.