ChatGPT is fun, but it is not funny! Humor is still challenging Large Language Models

• URL: http://arxiv.org/abs/2306.04563v1
• Date: Wed, 7 Jun 2023 16:10:21 GMT
• Title: ChatGPT is fun, but it is not funny! Humor is still challenging Large Language Models
• Authors: Sophie Jentzsch, Kristian Kersting
• Abstract summary: OpenAI's ChatGPT model almost seems to communicate on a human level and can even tell jokes. In a series of exploratory experiments around jokes, i.e., generation, explanation, and detection, we seek to understand ChatGPT's capability to grasp and reproduce human humor. Our empirical evidence indicates that jokes are not hard-coded but mostly also not newly generated by the model.
• Score: 19.399535453449488
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Humor is a central aspect of human communication that has not been solved for artificial agents so far. Large language models (LLMs) are increasingly able to capture implicit and contextual information. Especially, OpenAI's ChatGPT recently gained immense public attention. The GPT3-based model almost seems to communicate on a human level and can even tell jokes. Humor is an essential component of human communication. But is ChatGPT really funny? We put ChatGPT's sense of humor to the test. In a series of exploratory experiments around jokes, i.e., generation, explanation, and detection, we seek to understand ChatGPT's capability to grasp and reproduce human humor. Since the model itself is not accessible, we applied prompt-based experiments. Our empirical evidence indicates that jokes are not hard-coded but mostly also not newly generated by the model. Over 90% of 1008 generated jokes were the same 25 Jokes. The system accurately explains valid jokes but also comes up with fictional explanations for invalid jokes. Joke-typical characteristics can mislead ChatGPT in the classification of jokes. ChatGPT has not solved computational humor yet but it can be a big leap toward "funny" machines.

Related papers

• Getting Serious about Humor: Crafting Humor Datasets with Unfunny Large Language Models [27.936545041302377]
  Large language models (LLMs) can generate synthetic data for humor detection via editing texts. We benchmark LLMs on an existing human dataset and show that current LLMs display an impressive ability to 'unfun' jokes. We extend our approach to a code-mixed English-Hindi humor dataset, where we find that GPT-4's synthetic data is highly rated by bilingual annotators.
  arXiv  Detail & Related papers  (2024-02-23T02:58:12Z)
• Making Flow-Matching-Based Zero-Shot Text-to-Speech Laugh as You Like [49.2096391012794]
  ELaTE is a zero-shot TTS that can generate natural laughing speech of any speaker based on a short audio prompt. We develop our model based on the foundation of conditional flow-matching-based zero-shot TTS. We show that ELaTE can generate laughing speech with significantly higher quality and controllability compared to conventional models.
  arXiv  Detail & Related papers  (2024-02-12T02:58:10Z)
• To ChatGPT, or not to ChatGPT: That is the question! [78.407861566006]
  This study provides a comprehensive and contemporary assessment of the most recent techniques in ChatGPT detection. We have curated a benchmark dataset consisting of prompts from ChatGPT and humans, including diverse questions from medical, open Q&A, and finance domains. Our evaluation results demonstrate that none of the existing methods can effectively detect ChatGPT-generated content.
  arXiv  Detail & Related papers  (2023-04-04T03:04:28Z)
• Witscript: A System for Generating Improvised Jokes in a Conversation [0.0]
  Witscript is a novel joke generation system that can improvise original, contextually relevant jokes. Human evaluators judged Witscript's responses to input sentences to be jokes more than 40% of the time.
  arXiv  Detail & Related papers  (2023-02-03T21:30:34Z)
• The Naughtyformer: A Transformer Understands Offensive Humor [63.05016513788047]
  We introduce a novel jokes dataset filtered from Reddit and solve the subtype classification task using a finetuned Transformer dubbed the Naughtyformer. We show that our model is significantly better at detecting offensiveness in jokes compared to state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-11-25T20:37:58Z)
• ExPUNations: Augmenting Puns with Keywords and Explanations [88.58174386894913]
  We augment an existing dataset of puns with detailed crowdsourced annotations of keywords. This is the first humor dataset with such extensive and fine-grained annotations specifically for puns. We propose two tasks: explanation generation to aid with pun classification and keyword-conditioned pun generation.
  arXiv  Detail & Related papers  (2022-10-24T18:12:02Z)
• Towards Multimodal Prediction of Spontaneous Humour: A Novel Dataset and First Results [84.37263300062597]
  Humor is a substantial element of human social behavior, affect, and cognition. Current methods of humor detection have been exclusively based on staged data, making them inadequate for "real-world" applications. We contribute to addressing this deficiency by introducing the novel Passau-Spontaneous Football Coach Humor dataset, comprising about 11 hours of recordings.
  arXiv  Detail & Related papers  (2022-09-28T17:36:47Z)
• Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contest [70.40189243067857]
  Large neural networks can now generate jokes, but do they really "understand" humor? We challenge AI models with three tasks derived from the New Yorker Cartoon Caption Contest. We find that both types of models struggle at all three tasks.
  arXiv  Detail & Related papers  (2022-09-13T20:54:00Z)
• Uncertainty and Surprisal Jointly Deliver the Punchline: Exploiting Incongruity-Based Features for Humor Recognition [0.6445605125467573]
  We break down any joke into two distinct components: the set-up and the punchline. Inspired by the incongruity theory of humor, we model the set-up as the part developing semantic uncertainty. With increasingly powerful language models, we were able to feed the set-up along with the punchline into the GPT-2 language model.
  arXiv  Detail & Related papers  (2020-12-22T13:48:09Z)
• Dutch Humor Detection by Generating Negative Examples [5.888646114353371]
  Humor detection is usually modeled as a binary classification task, trained to predict if the given text is a joke or another type of text. We propose using text generation algorithms for imitating the original joke dataset to increase the difficulty for the learning algorithm. We compare the humor detection capabilities of classic neural network approaches with the state-of-the-art Dutch language model RobBERT.
  arXiv  Detail & Related papers  (2020-10-26T15:15:10Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.