Leveraging Large Language Models for Sarcastic Speech Annotation in Sarcasm Detection

• URL: http://arxiv.org/abs/2506.00955v1
• Date: Sun, 01 Jun 2025 11:00:18 GMT
• Title: Leveraging Large Language Models for Sarcastic Speech Annotation in Sarcasm Detection
• Authors: Zhu Li, Yuqing Zhang, Xiyuan Gao, Shekhar Nayak, Matt Coler,
• Abstract summary: Sarcasm fundamentally alters meaning through tone and context, yet detecting it in speech remains a challenge due to data scarcity.<n>We propose an annotation pipeline that leverages large language models (LLMs) to generate a sarcasm dataset.<n>We validate this approach by comparing annotation quality and detection performance on a publicly available sarcasm dataset.<n>Finally, we introduce PodSarc, a large-scale sarcastic speech dataset created through this pipeline.
• Score: 16.35106164874197
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sarcasm fundamentally alters meaning through tone and context, yet detecting it in speech remains a challenge due to data scarcity. In addition, existing detection systems often rely on multimodal data, limiting their applicability in contexts where only speech is available. To address this, we propose an annotation pipeline that leverages large language models (LLMs) to generate a sarcasm dataset. Using a publicly available sarcasm-focused podcast, we employ GPT-4o and LLaMA 3 for initial sarcasm annotations, followed by human verification to resolve disagreements. We validate this approach by comparing annotation quality and detection performance on a publicly available sarcasm dataset using a collaborative gating architecture. Finally, we introduce PodSarc, a large-scale sarcastic speech dataset created through this pipeline. The detection model achieves a 73.63% F1 score, demonstrating the dataset's potential as a benchmark for sarcasm detection research.

Related papers

• A Survey of Multimodal Sarcasm Detection [32.659528422756416]
  Sarcasm is a rhetorical device that is used to convey the opposite of the literal meaning of an utterance. We present the first comprehensive survey on multimodal sarcasm detection to date.
  arXiv  Detail & Related papers  (2024-10-24T16:17:47Z)
• KoCoSa: Korean Context-aware Sarcasm Detection Dataset [3.369750569233713]
  Sarcasm is a way of verbal irony where someone says the opposite of what they mean, often to ridicule a person, situation, or idea. In this paper, we introduce a new dataset for the Korean dialogue sarcasm detection task, KoCoSa. The dataset consists of 12.8K daily Korean dialogues and the labels for this task on the last response.
  arXiv  Detail & Related papers  (2024-02-22T10:17:57Z)
• Sentiment-enhanced Graph-based Sarcasm Explanation in Dialogue [63.32199372362483]
  We propose a novel sEntiment-enhanceD Graph-based multimodal sarcasm Explanation framework, named EDGE.<n>In particular, we first propose a lexicon-guided utterance sentiment inference module, where a utterance sentiment refinement strategy is devised.<n>We then develop a module named Joint Cross Attention-based Sentiment Inference (JCA-SI) by extending the multimodal sentiment analysis model JCA to derive the joint sentiment label for each video-audio clip.
  arXiv  Detail & Related papers  (2024-02-06T03:14:46Z)
• An Evaluation of State-of-the-Art Large Language Models for Sarcasm Detection [0.0]
  Sarcasm is the use of words by someone who means the opposite of what he is trying to say. Recent innovations in NLP have provided more possibilities for detecting sarcasm.
  arXiv  Detail & Related papers  (2023-10-07T14:45:43Z)
• Sarcasm Detection Framework Using Emotion and Sentiment Features [62.997667081978825]
  We propose a model which incorporates emotion and sentiment features to capture the incongruity intrinsic to sarcasm. Our approach achieved state-of-the-art results on four datasets from social networking platforms and online media.
  arXiv  Detail & Related papers  (2022-11-23T15:14:44Z)
• How to Describe Images in a More Funny Way? Towards a Modular Approach to Cross-Modal Sarcasm Generation [62.89586083449108]
  We study a new problem of cross-modal sarcasm generation (CMSG), i.e., generating a sarcastic description for a given image. CMSG is challenging as models need to satisfy the characteristics of sarcasm, as well as the correlation between different modalities. We propose an Extraction-Generation-Ranking based Modular method (EGRM) for cross-model sarcasm generation.
  arXiv  Detail & Related papers  (2022-11-20T14:38:24Z)
• Sign Language Recognition via Skeleton-Aware Multi-Model Ensemble [71.97020373520922]
  Sign language is commonly used by deaf or mute people to communicate. We propose a novel Multi-modal Framework with a Global Ensemble Model (GEM) for isolated Sign Language Recognition ( SLR) Our proposed SAM- SLR-v2 framework is exceedingly effective and achieves state-of-the-art performance with significant margins.
  arXiv  Detail & Related papers  (2021-10-12T16:57:18Z)
• Sarcasm Detection in Twitter -- Performance Impact when using Data Augmentation: Word Embeddings [0.0]
  Sarcasm is the use of words usually used to either mock or annoy someone, or for humorous purposes. We propose a contextual model for sarcasm identification in twitter using RoBERTa and augmenting the dataset. We achieve performance gain by 3.2% in the iSarcasm dataset when using data augmentation to increase 20% of data labeled as sarcastic.
  arXiv  Detail & Related papers  (2021-08-23T04:24:12Z)
• Augmenting Data for Sarcasm Detection with Unlabeled Conversation Context [55.898436183096614]
  We present a novel data augmentation technique, CRA (Contextual Response Augmentation), which utilizes conversational context to generate meaningful samples for training. Specifically, our proposed model, trained with the proposed data augmentation technique, participated in the sarcasm detection task of FigLang2020, have won and achieves the best performance in both Reddit and Twitter datasets.
  arXiv  Detail & Related papers  (2020-06-11T09:00:11Z)
• Sarcasm Detection using Context Separators in Online Discourse [3.655021726150369]
  Sarcasm is an intricate form of speech, where meaning is conveyed implicitly. In this work, we use RoBERTa_large to detect sarcasm in two datasets. We also assert the importance of context in improving the performance of contextual word embedding models.
  arXiv  Detail & Related papers  (2020-06-01T10:52:35Z)
• $R^3$: Reverse, Retrieve, and Rank for Sarcasm Generation with Commonsense Knowledge [51.70688120849654]
  We propose an unsupervised approach for sarcasm generation based on a non-sarcastic input sentence. Our method employs a retrieve-and-edit framework to instantiate two major characteristics of sarcasm.
  arXiv  Detail & Related papers  (2020-04-28T02:30:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.