Can LLMs Simulate Social Media Engagement? A Study on Action-Guided Response Generation

• URL: http://arxiv.org/abs/2502.12073v1
• Date: Mon, 17 Feb 2025 17:43:08 GMT
• Title: Can LLMs Simulate Social Media Engagement? A Study on Action-Guided Response Generation
• Authors: Zhongyi Qiu, Hanjia Lyu, Wei Xiong, Jiebo Luo,
• Abstract summary: This paper analyzes large language models' ability to simulate social media engagement through action guided response generation.<n>We benchmark GPT-4o-mini, O1-mini, and DeepSeek-R1 in social media engagement simulation regarding a major societal event.
• Score: 51.44040615856536
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Social media enables dynamic user engagement with trending topics, and recent research has explored the potential of large language models (LLMs) for response generation. While some studies investigate LLMs as agents for simulating user behavior on social media, their focus remains on practical viability and scalability rather than a deeper understanding of how well LLM aligns with human behavior. This paper analyzes LLMs' ability to simulate social media engagement through action guided response generation, where a model first predicts a user's most likely engagement action-retweet, quote, or rewrite-towards a trending post before generating a personalized response conditioned on the predicted action. We benchmark GPT-4o-mini, O1-mini, and DeepSeek-R1 in social media engagement simulation regarding a major societal event discussed on X. Our findings reveal that zero-shot LLMs underperform BERT in action prediction, while few-shot prompting initially degrades the prediction accuracy of LLMs with limited examples. However, in response generation, few-shot LLMs achieve stronger semantic alignment with ground truth posts.

Related papers

• LLM Social Simulations Are a Promising Research Method [4.6456873975541635]
  We argue that the promise of large language model (LLM) social simulations can be achieved by addressing five tractable challenges. We believe that LLM social simulations can already be used for exploratory research, such as pilot experiments for psychology, economics, sociology, and marketing.
  arXiv  Detail & Related papers  (2025-04-03T03:01:26Z)
• LLM Agents That Act Like Us: Accurate Human Behavior Simulation with Real-World Data [26.506531028553795]
  Recent research shows that LLMs can simulate believable'' human behaviors to power LLM agents via prompt-only methods. We focus on evaluating and improving LLM's objective accuracy'' rather than the subjective believability'' in the web action generation task. We present the first comprehensive quantitative evaluation of state-of-the-art LLMs on the task of web action generation.
  arXiv  Detail & Related papers  (2025-03-26T17:33:27Z)
• Bayesian Teaching Enables Probabilistic Reasoning in Large Language Models [50.16340812031201]
  We show that large language models (LLMs) do not update their beliefs as expected from the Bayesian framework. We teach the LLMs to reason in a Bayesian manner by training them to mimic the predictions of an optimal Bayesian model.
  arXiv  Detail & Related papers  (2025-03-21T20:13:04Z)
• Engagement-Driven Content Generation with Large Language Models [8.049552839071918]
  Large Language Models (LLMs) exhibit significant persuasion capabilities in one-on-one interactions. This study investigates the potential social impact of LLMs in interconnected users and complex opinion dynamics.
  arXiv  Detail & Related papers  (2024-11-20T10:40:08Z)
• GenSim: A General Social Simulation Platform with Large Language Model based Agents [111.00666003559324]
  We propose a novel large language model (LLMs)-based simulation platform called textitGenSim. Our platform supports one hundred thousand agents to better simulate large-scale populations in real-world contexts. To our knowledge, GenSim represents an initial step toward a general, large-scale, and correctable social simulation platform.
  arXiv  Detail & Related papers  (2024-10-06T05:02:23Z)
• Are Large Language Models (LLMs) Good Social Predictors? [36.68104332805214]
  We show that Large Language Models (LLMs) cannot work as expected on social prediction when given general input features without shortcuts. We introduce a novel social prediction task, Soc-PRF Prediction, which utilizes general features as input and simulates real-world social study settings.
  arXiv  Detail & Related papers  (2024-02-20T00:59:22Z)
• Large Language Models: A Survey [69.72787936480394]
  Large Language Models (LLMs) have drawn a lot of attention due to their strong performance on a wide range of natural language tasks. LLMs' ability of general-purpose language understanding and generation is acquired by training billions of model's parameters on massive amounts of text data.
  arXiv  Detail & Related papers  (2024-02-09T05:37:09Z)
• Learning to Generate Explainable Stock Predictions using Self-Reflective Large Language Models [54.21695754082441]
  We propose a framework to teach Large Language Models (LLMs) to generate explainable stock predictions. A reflective agent learns how to explain past stock movements through self-reasoning, while the PPO trainer trains the model to generate the most likely explanations. Our framework can outperform both traditional deep-learning and LLM methods in prediction accuracy and Matthews correlation coefficient.
  arXiv  Detail & Related papers  (2024-02-06T03:18:58Z)
• Do LLM Agents Exhibit Social Behavior? [5.094340963261968]
  State-Understanding-Value-Action (SUVA) is a framework to systematically analyze responses in social contexts. It assesses social behavior through both their final decisions and the response generation processes leading to those decisions. We demonstrate that utterance-based reasoning reliably predicts LLMs' final actions.
  arXiv  Detail & Related papers  (2023-12-23T08:46:53Z)
• On Learning to Summarize with Large Language Models as References [101.79795027550959]
  Large language models (LLMs) are favored by human annotators over the original reference summaries in commonly used summarization datasets. We study an LLM-as-reference learning setting for smaller text summarization models to investigate whether their performance can be substantially improved.
  arXiv  Detail & Related papers  (2023-05-23T16:56:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.