Spoken in Jest, Detected in Earnest: A Systematic Review of Sarcasm Recognition -- Multimodal Fusion, Challenges, and Future Prospects

• URL: http://arxiv.org/abs/2509.04605v1
• Date: Thu, 04 Sep 2025 18:40:52 GMT
• Title: Spoken in Jest, Detected in Earnest: A Systematic Review of Sarcasm Recognition -- Multimodal Fusion, Challenges, and Future Prospects
• Authors: Xiyuan Gao, Shekhar Nayak, Matt Coler,
• Abstract summary: Sarcasm, a common feature of human communication, poses challenges in interpersonal interactions and human-machine interactions.<n>Recent advancements in speech technology emphasize the growing importance of leveraging speech data for automatic sarcasm recognition.<n>This systematic review is the first to focus on speech-based sarcasm recognition, charting the evolution from unimodal to multimodal approaches.
• Score: 9.844842556762067
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Sarcasm, a common feature of human communication, poses challenges in interpersonal interactions and human-machine interactions. Linguistic research has highlighted the importance of prosodic cues, such as variations in pitch, speaking rate, and intonation, in conveying sarcastic intent. Although previous work has focused on text-based sarcasm detection, the role of speech data in recognizing sarcasm has been underexplored. Recent advancements in speech technology emphasize the growing importance of leveraging speech data for automatic sarcasm recognition, which can enhance social interactions for individuals with neurodegenerative conditions and improve machine understanding of complex human language use, leading to more nuanced interactions. This systematic review is the first to focus on speech-based sarcasm recognition, charting the evolution from unimodal to multimodal approaches. It covers datasets, feature extraction, and classification methods, and aims to bridge gaps across diverse research domains. The findings include limitations in datasets for sarcasm recognition in speech, the evolution of feature extraction techniques from traditional acoustic features to deep learning-based representations, and the progression of classification methods from unimodal approaches to multimodal fusion techniques. In so doing, we identify the need for greater emphasis on cross-cultural and multilingual sarcasm recognition, as well as the importance of addressing sarcasm as a multimodal phenomenon, rather than a text-based challenge.

Related papers

• Integrating Feedback Loss from Bi-modal Sarcasm Detector for Sarcastic Speech Synthesis [14.798970809585066]
  Sarcastic speech synthesis is essential for enhancing natural interactions in applications such as entertainment and human-computer interaction.<n>This study introduces a novel approach that integrates feedback loss from a bi-modal sarcasm detection model into the TTS training process.
  arXiv  Detail & Related papers  (2025-08-18T15:44:54Z)
• Developmental Predictive Coding Model for Early Infancy Mono and Bilingual Vocal Continual Learning [69.8008228833895]
  We propose a small-sized generative neural network equipped with a continual learning mechanism.<n>Our model prioritizes interpretability and demonstrates the advantages of online learning.
  arXiv  Detail & Related papers  (2024-12-23T10:23:47Z)
• RCLMuFN: Relational Context Learning and Multiplex Fusion Network for Multimodal Sarcasm Detection [1.023096557577223]
  We propose a relational context learning and multiplex fusion network (RCLMuFN) for multimodal sarcasm detection.<n> Firstly, we employ four feature extractors to comprehensively extract features from raw text and images.<n> Secondly, we utilize the relational context learning module to learn the contextual information of text and images.
  arXiv  Detail & Related papers  (2024-12-17T15:29:31Z)
• Multimodal Fusion with LLMs for Engagement Prediction in Natural Conversation [70.52558242336988]
  We focus on predicting engagement in dyadic interactions by scrutinizing verbal and non-verbal cues, aiming to detect signs of disinterest or confusion. In this work, we collect a dataset featuring 34 participants engaged in casual dyadic conversations, each providing self-reported engagement ratings at the end of each conversation. We introduce a novel fusion strategy using Large Language Models (LLMs) to integrate multiple behavior modalities into a multimodal transcript''
  arXiv  Detail & Related papers  (2024-09-13T18:28:12Z)
• Sentiment-enhanced Graph-based Sarcasm Explanation in Dialogue [63.32199372362483]
  We propose a novel sEntiment-enhanceD Graph-based multimodal sarcasm Explanation framework, named EDGE.<n>In particular, we first propose a lexicon-guided utterance sentiment inference module, where a utterance sentiment refinement strategy is devised.<n>We then develop a module named Joint Cross Attention-based Sentiment Inference (JCA-SI) by extending the multimodal sentiment analysis model JCA to derive the joint sentiment label for each video-audio clip.
  arXiv  Detail & Related papers  (2024-02-06T03:14:46Z)
• Recent Advances in Hate Speech Moderation: Multimodality and the Role of Large Models [52.24001776263608]
  This comprehensive survey delves into the recent strides in HS moderation. We highlight the burgeoning role of large language models (LLMs) and large multimodal models (LMMs) We identify existing gaps in research, particularly in the context of underrepresented languages and cultures.
  arXiv  Detail & Related papers  (2024-01-30T03:51:44Z)
• Towards Multi-Modal Sarcasm Detection via Hierarchical Congruity Modeling with Knowledge Enhancement [31.97249246223621]
  Sarcasm is a linguistic phenomenon indicating a discrepancy between literal meanings and implied intentions. Most existing techniques only modeled the atomic-level inconsistencies between the text input and its accompanying image. We propose a novel hierarchical framework for sarcasm detection by exploring both the atomic-level congruity based on multi-head cross attention mechanism and the composition-level congruity based on graph neural networks.
  arXiv  Detail & Related papers  (2022-10-07T12:44:33Z)
• Co-Located Human-Human Interaction Analysis using Nonverbal Cues: A Survey [71.43956423427397]
  We aim to identify the nonverbal cues and computational methodologies resulting in effective performance. This survey differs from its counterparts by involving the widest spectrum of social phenomena and interaction settings. Some major observations are: the most often used nonverbal cue, computational method, interaction environment, and sensing approach are speaking activity, support vector machines, and meetings composed of 3-4 persons equipped with microphones and cameras, respectively.
  arXiv  Detail & Related papers  (2022-07-20T13:37:57Z)
• Multi-Modal Sarcasm Detection Based on Contrastive Attention Mechanism [7.194040730138362]
  We construct a Contras-tive-Attention-based Sarcasm Detection (ConAttSD) model, which uses an inter-modality contrastive attention mechanism to extract contrastive features for an utterance. Our experiments on MUStARD, a benchmark multi-modal sarcasm dataset, demonstrate the effectiveness of the proposed ConAttSD model.
  arXiv  Detail & Related papers  (2021-09-30T14:17:51Z)
• Interpretable Multi-Head Self-Attention model for Sarcasm Detection in social media [0.0]
  Inherent ambiguity in sarcastic expressions, make sarcasm detection very difficult. We develop an interpretable deep learning model using multi-head self-attention and gated recurrent units. We show the effectiveness of our approach by achieving state-of-the-art results on multiple datasets.
  arXiv  Detail & Related papers  (2021-01-14T21:39:35Z)
• You Impress Me: Dialogue Generation via Mutual Persona Perception [62.89449096369027]
  The research in cognitive science suggests that understanding is an essential signal for a high-quality chit-chat conversation. Motivated by this, we propose P2 Bot, a transmitter-receiver based framework with the aim of explicitly modeling understanding.
  arXiv  Detail & Related papers  (2020-04-11T12:51:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.