SMILE-Next: Teaching Large Language Models to Detect, Classify, and Reason about Laughter

• URL: http://arxiv.org/abs/2605.28084v1
• Date: Wed, 27 May 2026 07:39:37 GMT
• Title: SMILE-Next: Teaching Large Language Models to Detect, Classify, and Reason about Laughter
• Abstract summary: We introduce SMILE-Next, a dataset for real-world laughter understanding with multimodal textual representations and question-answer annotations.<n>Building upon SMILE-Next, we aim to develop a laughter-specialized large language model capable of nuanced understanding of laughter in real-world contexts.<n>We propose two key components: laughter-specific Self-Instruct and the Mixture-of-Laugh-Experts framework.
• Score: 26.23604565651714
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Laughter is a complex social signal that conveys communicative intent beyond amusement. While prior work has focused on isolated laughter analysis tasks, a comprehensive understanding of laughter in real-world scenarios remains underexplored. Therefore, we introduce SMILE-Next, a dataset for real-world laughter understanding with multimodal textual representations and question-answer annotations across three tasks: laughter detection, laughter type classification, and laughter reasoning. Building upon SMILE-Next, we aim to develop a laughter-specialized large language model capable of nuanced understanding of laughter in real-world contexts. To this end, we propose two key components: laughter-specific Self-Instruct and the Mixture-of-Laugh-Experts (MoLE) framework. Laughter-specific Self-Instruct enhances generalization across tasks and domains by automatically synthesizing diverse laughter-centric instructions. MoLE introduces a task-adaptive expert routing mechanism that dynamically selects specialized experts tailored to each laughter-related task, improving task-specific performance and efficiency. Experimental results show that the combination of our proposed components substantially outperforms multimodal LLM baselines, advancing robust real-world laughter understanding. Project page is at: https://mok0102.github.io/smile-next/.

Related papers

• On the Wings of Imagination: Conflicting Script-based Multi-role Framework for Humor Caption Generation [10.157232656580659]
  Humor is a commonly used and intricate human language in daily life.<n>We develop a novel humor generation mechanism based on a fundamental humor theory, GTVH.<n>To produce funny and script-opposite captions, we introduce a humor-theory-driven multi-role LLM collaboration framework.
  arXiv  Detail & Related papers  (2026-02-06T06:41:33Z)
• V-HUB: A Visual-Centric Humor Understanding Benchmark for Video LLMs [72.59885036868499]
  v-HUB is a visual-centric video humor understanding benchmark.<n>Each video clip is paired with rich annotations, including captions, descriptions, and explanations.<n>We evaluate a diverse set of MLLMs, from specialized Video-LLMs to versatile OmniLLMs that can process audio.
  arXiv  Detail & Related papers  (2025-09-30T04:33:52Z)
• Why Do We Laugh? Annotation and Taxonomy Generation for Laughable Contexts in Spontaneous Text Conversation [22.249086790195634]
  This study developed a taxonomy to classify the underlying reasons for laughable contexts.<n>It also evaluated GPT-4o's performance in recognizing the majority labels of laughable contexts, achieving an F1 score of 43.14%.<n>These findings contribute to the advancement of conversational AI by establishing a foundation for more nuanced recognition and generation of laughter.
  arXiv  Detail & Related papers  (2025-01-28T02:16:18Z)
• Text Is Not All You Need: Multimodal Prompting Helps LLMs Understand Humor [0.0]
  Humor is frequently multimodal, relying on phonetic ambiguity, rhythm and timing to convey meaning.<n>We present an LLM with both the text and the spoken form of a joke, generated using an off-the-shelf text-to-speech (TTS) system.
  arXiv  Detail & Related papers  (2024-12-01T06:49:31Z)
• Can Pre-trained Language Models Understand Chinese Humor? [74.96509580592004]
  This paper is the first work that systematically investigates the humor understanding ability of pre-trained language models (PLMs) We construct a comprehensive Chinese humor dataset, which can fully meet all the data requirements of the proposed evaluation framework. Our empirical study on the Chinese humor dataset yields some valuable observations, which are of great guiding value for future optimization of PLMs in humor understanding and generation.
  arXiv  Detail & Related papers  (2024-07-04T18:13:38Z)
• Reasoning in Conversation: Solving Subjective Tasks through Dialogue Simulation for Large Language Models [56.93074140619464]
  We propose RiC (Reasoning in Conversation), a method that focuses on solving subjective tasks through dialogue simulation. The motivation of RiC is to mine useful contextual information by simulating dialogues instead of supplying chain-of-thought style rationales. We evaluate both API-based and open-source LLMs including GPT-4, ChatGPT, and OpenChat across twelve tasks.
  arXiv  Detail & Related papers  (2024-02-27T05:37:10Z)
• Making Flow-Matching-Based Zero-Shot Text-to-Speech Laugh as You Like [49.2096391012794]
  ELaTE is a zero-shot TTS that can generate natural laughing speech of any speaker based on a short audio prompt. We develop our model based on the foundation of conditional flow-matching-based zero-shot TTS. We show that ELaTE can generate laughing speech with significantly higher quality and controllability compared to conventional models.
  arXiv  Detail & Related papers  (2024-02-12T02:58:10Z)
• SMILE: Multimodal Dataset for Understanding Laughter in Video with Language Models [32.60274453610208]
  We tackle a new challenge for machines to understand the rationale behind laughter in video. Our proposed dataset, SMILE, comprises video clips and language descriptions of why people laugh.
  arXiv  Detail & Related papers  (2023-12-15T14:17:45Z)
• SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Models [86.478087039015]
  We present a versatile multi-modal large language model (MLLM) with a joint mixing of model weights, tuning tasks, and visual embeddings. Based on our proposed joint mixing, we propose an efficient strategy aiming to better capture fine-grained appearances of high-resolution images. We hope our work may cast a light on the exploration of joint mixing in future MLLM research.
  arXiv  Detail & Related papers  (2023-11-13T18:59:47Z)
• D$^2$TV: Dual Knowledge Distillation and Target-oriented Vision Modeling for Many-to-Many Multimodal Summarization [113.72253589338472]
  Many-to-many multimodal summarization (M$3$S) task aims to generate summaries in any language with document inputs in any language and the corresponding image sequence. We propose a dual knowledge distillation and target-oriented vision modeling framework for the M$3$S task.
  arXiv  Detail & Related papers  (2023-05-22T06:47:35Z)
• Laughing Matters: Introducing Laughing-Face Generation using Diffusion Models [35.688696422879175]
  We propose a novel model capable of generating realistic laughter sequences, given a still portrait and an audio clip containing laughter. We train our model on a diverse set of laughter datasets and introduce an evaluation metric specifically designed for laughter. Our model achieves state-of-the-art performance across all metrics, even when these are re-trained for laughter generation.
  arXiv  Detail & Related papers  (2023-05-15T17:59:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.