Pun Unintended: LLMs and the Illusion of Humor Understanding

• URL: http://arxiv.org/abs/2509.12158v2
• Date: Sat, 20 Sep 2025 12:16:33 GMT
• Title: Pun Unintended: LLMs and the Illusion of Humor Understanding
• Authors: Alessandro Zangari, Matteo Marcuzzo, Andrea Albarelli, Mohammad Taher Pilehvar, Jose Camacho-Collados,
• Abstract summary: Puns are a form of humorous wordplay that exploits polysemy and phonetic similarity.<n>Our contributions include comprehensive and nuanced pun detection benchmarks, human evaluation across recent LLMs, and an analysis of the robustness challenges these models face in processing puns.
• Score: 50.29407048003165
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Puns are a form of humorous wordplay that exploits polysemy and phonetic similarity. While LLMs have shown promise in detecting puns, we show in this paper that their understanding often remains shallow, lacking the nuanced grasp typical of human interpretation. By systematically analyzing and reformulating existing pun benchmarks, we demonstrate how subtle changes in puns are sufficient to mislead LLMs. Our contributions include comprehensive and nuanced pun detection benchmarks, human evaluation across recent LLMs, and an analysis of the robustness challenges these models face in processing puns.

Related papers

• Engagement Undermines Safety: How Stereotypes and Toxicity Shape Humor in Language Models [55.98686105081078]
  Large language models are increasingly used for creative writing and engagement content, raising safety concerns about the outputs.<n>This work evaluates how funniness optimization in modern LLM pipelines couples with harmful content by measuring humor, stereotypicality, and toxicity.
  arXiv  Detail & Related papers  (2025-10-21T09:28:09Z)
• Comparing Apples to Oranges: A Dataset & Analysis of LLM Humour Understanding from Traditional Puns to Topical Jokes [21.197328006274578]
  We compare models' joke explanation abilities from simple puns to complex topical humour.<n>To this end, we curate a dataset of 600 jokes across 4 joke types.<n>These jokes include heterographic and homographic puns, contemporary internet humour, and topical jokes.
  arXiv  Detail & Related papers  (2025-07-17T17:51:20Z)
• "A good pun is its own reword": Can Large Language Models Understand Puns? [9.541689402830642]
  Puns play a vital role in academic research due to their distinct structure and clear definition. The understanding of puns in large language models (LLMs) has not been thoroughly examined.
  arXiv  Detail & Related papers  (2024-04-21T09:42:05Z)
• Fantastic Semantics and Where to Find Them: Investigating Which Layers of Generative LLMs Reflect Lexical Semantics [50.982315553104975]
  We investigate the bottom-up evolution of lexical semantics for a popular large language model, namely Llama2. Our experiments show that the representations in lower layers encode lexical semantics, while the higher layers, with weaker semantic induction, are responsible for prediction. This is in contrast to models with discriminative objectives, such as mask language modeling, where the higher layers obtain better lexical semantics.
  arXiv  Detail & Related papers  (2024-03-03T13:14:47Z)
• Context-Situated Pun Generation [42.727010784168115]
  We propose a new task, context-situated pun generation, where a specific context represented by a set of keywords is provided. The task is to first identify suitable pun words that are appropriate for the context, then generate puns based on the context keywords and the identified pun words. We show that 69% of our top retrieved pun words can be used to generate context-situated puns, and our generation module yields successful 31% of the time.
  arXiv  Detail & Related papers  (2022-10-24T18:24:48Z)
• ExPUNations: Augmenting Puns with Keywords and Explanations [88.58174386894913]
  We augment an existing dataset of puns with detailed crowdsourced annotations of keywords. This is the first humor dataset with such extensive and fine-grained annotations specifically for puns. We propose two tasks: explanation generation to aid with pun classification and keyword-conditioned pun generation.
  arXiv  Detail & Related papers  (2022-10-24T18:12:02Z)
• AmbiPun: Generating Humorous Puns with Ambiguous Context [31.81213062995652]
  Our model first produces a list of related concepts through a reverse dictionary. We then utilize one-shot GPT3 to generate context words and then generate puns incorporating context words from both concepts. Human evaluation shows that our method successfully generates pun 52% of the time.
  arXiv  Detail & Related papers  (2022-05-04T00:24:11Z)
• "The Boating Store Had Its Best Sail Ever": Pronunciation-attentive Contextualized Pun Recognition [80.59427655743092]
  We propose Pronunciation-attentive Contextualized Pun Recognition (PCPR) to perceive human humor. PCPR derives contextualized representation for each word in a sentence by capturing the association between the surrounding context and its corresponding phonetic symbols. Results demonstrate that the proposed approach significantly outperforms the state-of-the-art methods in pun detection and location tasks.
  arXiv  Detail & Related papers  (2020-04-29T20:12:20Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.