Do We Trust What They Say or What They Do? A Multimodal User Embedding Provides Personalized Explanations

• URL: http://arxiv.org/abs/2409.02965v1
• Date: Wed, 4 Sep 2024 02:17:32 GMT
• Title: Do We Trust What They Say or What They Do? A Multimodal User Embedding Provides Personalized Explanations
• Authors: Zhicheng Ren, Zhiping Xiao, Yizhou Sun,
• Abstract summary: We propose Contribution-Aware Multimodal User Embedding (CAMUE) for social networks. We show that our approach can provide personalized explainable predictions, automatically mitigating the impact of unreliable information. Our work paves the way for more explainable, reliable, and effective social media user embedding.
• Score: 35.77028281332307
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: With the rapid development of social media, the importance of analyzing social network user data has also been put on the agenda. User representation learning in social media is a critical area of research, based on which we can conduct personalized content delivery, or detect malicious actors. Being more complicated than many other types of data, social network user data has inherent multimodal nature. Various multimodal approaches have been proposed to harness both text (i.e. post content) and relation (i.e. inter-user interaction) information to learn user embeddings of higher quality. The advent of Graph Neural Network models enables more end-to-end integration of user text embeddings and user interaction graphs in social networks. However, most of those approaches do not adequately elucidate which aspects of the data - text or graph structure information - are more helpful for predicting each specific user under a particular task, putting some burden on personalized downstream analysis and untrustworthy information filtering. We propose a simple yet effective framework called Contribution-Aware Multimodal User Embedding (CAMUE) for social networks. We have demonstrated with empirical evidence, that our approach can provide personalized explainable predictions, automatically mitigating the impact of unreliable information. We also conducted case studies to show how reasonable our results are. We observe that for most users, graph structure information is more trustworthy than text information, but there are some reasonable cases where text helps more. Our work paves the way for more explainable, reliable, and effective social media user embedding which allows for better personalized content delivery.

Related papers

• SoMeR: Multi-View User Representation Learning for Social Media [1.7949335303516192]
  We propose SoMeR, a Social Media user representation learning framework that incorporates temporal activities, text content, profile information, and network interactions to learn comprehensive user portraits. SoMeR encodes user post streams as sequences of timestamped textual features, uses transformers to embed this along with profile data, and jointly trains with link prediction and contrastive learning objectives. We demonstrate SoMeR's versatility through two applications: 1) Identifying inauthentic accounts involved in coordinated influence operations by detecting users posting similar content simultaneously, and 2) Measuring increased polarization in online discussions after major events by quantifying how users with different beliefs moved farther apart
  arXiv  Detail & Related papers  (2024-05-02T22:26:55Z)
• Decoding the Silent Majority: Inducing Belief Augmented Social Graph with Large Language Model for Response Forecasting [74.68371461260946]
  SocialSense is a framework that induces a belief-centered graph on top of an existent social network, along with graph-based propagation to capture social dynamics. Our method surpasses existing state-of-the-art in experimental evaluations for both zero-shot and supervised settings.
  arXiv  Detail & Related papers  (2023-10-20T06:17:02Z)
• Cross-Network Social User Embedding with Hybrid Differential Privacy Guarantees [81.6471440778355]
  We propose a Cross-network Social User Embedding framework, namely DP-CroSUE, to learn the comprehensive representations of users in a privacy-preserving way. In particular, for each heterogeneous social network, we first introduce a hybrid differential privacy notion to capture the variation of privacy expectations for heterogeneous data types. To further enhance user embeddings, a novel cross-network GCN embedding model is designed to transfer knowledge across networks through those aligned users.
  arXiv  Detail & Related papers  (2022-09-04T06:22:37Z)
• Personalized multi-faceted trust modeling to determine trust links in social media and its potential for misinformation management [61.88858330222619]
  We present an approach for predicting trust links between peers in social media. We propose a data-driven multi-faceted trust modeling which incorporates many distinct features for a comprehensive analysis. Illustrated in a trust-aware item recommendation task, we evaluate the proposed framework in the context of a large Yelp dataset.
  arXiv  Detail & Related papers  (2021-11-11T19:40:51Z)
• Two-Faced Humans on Twitter and Facebook: Harvesting Social Multimedia for Human Personality Profiling [74.83957286553924]
  We infer the Myers-Briggs Personality Type indicators by applying a novel multi-view fusion framework, called "PERS" Our experimental results demonstrate the PERS's ability to learn from multi-view data for personality profiling by efficiently leveraging on the significantly different data arriving from diverse social multimedia sources.
  arXiv  Detail & Related papers  (2021-06-20T10:48:49Z)
• Learning User Embeddings from Temporal Social Media Data: A Survey [15.324014759254915]
  We survey representative work on learning a concise latent user representation (a.k.a. user embedding) that can capture the main characteristics of a social media user. The learned user embeddings can later be used to support different downstream user analysis tasks such as personality modeling, suicidal risk assessment and purchase decision prediction.
  arXiv  Detail & Related papers  (2021-05-17T16:22:43Z)
• Analysis of Social Media Data using Multimodal Deep Learning for Disaster Response [6.8889797054846795]
  We propose to use both text and image modalities of social media data to learn a joint representation using state-of-the-art deep learning techniques. Experiments on real-world disaster datasets show that the proposed multimodal architecture yields better performance than models trained using a single modality.
  arXiv  Detail & Related papers  (2020-04-14T19:36:11Z)
• I Know Where You Are Coming From: On the Impact of Social Media Sources on AI Model Performance [79.05613148641018]
  We will study the performance of different machine learning models when being learned on multi-modal data from different social networks. Our initial experimental results reveal that social network choice impacts the performance.
  arXiv  Detail & Related papers  (2020-02-05T11:10:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.