Impact of Stickers on Multimodal Chat Sentiment Analysis and Intent Recognition: A New Task, Dataset and Baseline

• URL: http://arxiv.org/abs/2405.08427v1
• Date: Tue, 14 May 2024 08:42:49 GMT
• Title: Impact of Stickers on Multimodal Chat Sentiment Analysis and Intent Recognition: A New Task, Dataset and Baseline
• Authors: Yuanchen Shi, Biao Ma, Fang Kong,
• Abstract summary: We propose a new task: Multimodal chat Sentiment Analysis and Intent Recognition involving Stickers (MSAIRS) We introduce a novel multimodal dataset containing Chinese chat records and stickers excerpted from several mainstream social media platforms. Our dataset and code will be publicly available.
• Score: 4.375392069380812
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Stickers are increasingly used in social media to express sentiment and intent. When finding typing troublesome, people often use a sticker instead. Despite the significant impact of stickers on sentiment analysis and intent recognition, little research has been conducted. To address this gap, we propose a new task: Multimodal chat Sentiment Analysis and Intent Recognition involving Stickers (MSAIRS). Additionally, we introduce a novel multimodal dataset containing Chinese chat records and stickers excerpted from several mainstream social media platforms. Our dataset includes paired data with the same text but different stickers, and various stickers consisting of the same images with different texts, allowing us to better understand the impact of stickers on chat sentiment and intent. We also propose an effective multimodal joint model, MMSAIR, for our task, which is validated on our datasets and indicates that visual information of stickers counts. Our dataset and code will be publicly available.

Related papers

• Multi-modal Stance Detection: New Datasets and Model [56.97470987479277]
  We study multi-modal stance detection for tweets consisting of texts and images. We propose a simple yet effective Targeted Multi-modal Prompt Tuning framework (TMPT) TMPT achieves state-of-the-art performance in multi-modal stance detection.
  arXiv  Detail & Related papers  (2024-02-22T05:24:19Z)
• Sticker820K: Empowering Interactive Retrieval with Stickers [34.67442172774095]
  We propose a large-scale Chinese sticker dataset, namely Sticker820K, which consists of 820k image-text pairs. Each sticker has rich and high-quality textual annotations, including descriptions, optical characters, emotional labels, and style classifications. For the text-to-image retrieval task, our StickerCLIP demonstrates strong superiority over the CLIP, which achieves an absolute gain of 66.0% in mean recall.
  arXiv  Detail & Related papers  (2023-06-12T05:06:53Z)
• Micro-video Tagging via Jointly Modeling Social Influence and Tag Relation [56.23157334014773]
  85.7% of micro-videos lack annotation. Existing methods mostly focus on analyzing video content, neglecting users' social influence and tag relation. We formulate micro-video tagging as a link prediction problem in a constructed heterogeneous network.
  arXiv  Detail & Related papers  (2023-03-15T02:13:34Z)
• Selecting Stickers in Open-Domain Dialogue through Multitask Learning [51.67855506570727]
  We propose a multitask learning method comprised of three auxiliary tasks to enhance the understanding of dialogue history, emotion and semantic meaning of stickers. Our model can better combine the multimodal information and achieve significantly higher accuracy over strong baselines.
  arXiv  Detail & Related papers  (2022-09-16T03:45:22Z)
• Learning to Respond with Your Favorite Stickers: A Framework of Unifying Multi-Modality and User Preference in Multi-Turn Dialog [67.91114640314004]
  Stickers with vivid and engaging expressions are becoming increasingly popular in online messaging apps. Some works are dedicated to automatically select sticker response by matching the stickers image with previous utterances. We propose to recommend an appropriate sticker to user based on multi-turn dialog context and sticker using history of user.
  arXiv  Detail & Related papers  (2020-11-05T03:31:17Z)
• Content-based Analysis of the Cultural Differences between TikTok and Douyin [95.32409577885645]
  Short-form video social media shifts away from the traditional media paradigm by telling the audience a dynamic story to attract their attention. In particular, different combinations of everyday objects can be employed to represent a unique scene that is both interesting and understandable. Offered by the same company, TikTok and Douyin are popular examples of such new media that has become popular in recent years. The hypothesis that they express cultural differences together with media fashion and social idiosyncrasy is the primary target of our research.
  arXiv  Detail & Related papers  (2020-11-03T01:47:49Z)
• Learning to Respond with Stickers: A Framework of Unifying Multi-Modality in Multi-Turn Dialog [65.7021675527543]
  Stickers with vivid and engaging expressions are becoming increasingly popular in online messaging apps. Some works are dedicated to automatically select sticker response by matching text labels of stickers with previous utterances. We propose to recommend an appropriate sticker to user based on multi-turn dialog context history without any external labels.
  arXiv  Detail & Related papers  (2020-03-10T13:10:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.