Debiasing Large Language Models toward Social Factors in Online Behavior Analytics through Prompt Knowledge Tuning

• URL: http://arxiv.org/abs/2603.27057v1
• Date: Sat, 28 Mar 2026 00:15:40 GMT
• Title: Debiasing Large Language Models toward Social Factors in Online Behavior Analytics through Prompt Knowledge Tuning
• Abstract summary: Attribution theory explains how individuals attribute others' behavior in a social context by employing personal (dispositional) and impersonal (situational) causality.<n>Large Language Models (LLMs), trained on human-generated corpora, may implicitly mimic this social attribution process in social contexts.<n>We introduce a scalable method to mitigate such biases by enriching the instruction prompts for LLMs with two prompt aids using social-attribution knowledge.
• Score: 1.357291726431012
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Attribution theory explains how individuals interpret and attribute others' behavior in a social context by employing personal (dispositional) and impersonal (situational) causality. Large Language Models (LLMs), trained on human-generated corpora, may implicitly mimic this social attribution process in social contexts. However, the extent to which LLMs utilize these causal attributions in their reasoning remains underexplored. Although using reasoning paradigms, such as Chain-of-Thought (CoT), has shown promising results in various tasks, ignoring social attribution in reasoning could lead to biased responses by LLMs in social contexts. In this study, we investigate the impact of incorporating a user's goal as knowledge to infer dispositional causality and message context to infer situational causality on LLM performance. To this end, we introduce a scalable method to mitigate such biases by enriching the instruction prompts for LLMs with two prompt aids using social-attribution knowledge, based on the context and goal of a social media message. This method improves the model performance while reducing the social-attribution bias of the LLM in the reasoning on zero-shot classification tasks for behavior analytics applications. We empirically show the benefits of our method across two tasks-intent detection and theme detection on social media in the disaster domain-when considering the variability of disaster types and multiple languages of social media. Our experiments highlight the biases of three open-source LLMs: Llama3, Mistral, and Gemma, toward social attribution, and show the effectiveness of our mitigation strategies.

Related papers

• MARO: Learning Stronger Reasoning from Social Interaction [7.77506109184819]
  Multi-Agent Reward Optimization (MARO) is a method that enables large language models to acquire stronger reasoning abilities.<n> Experimental results demonstrate that MARO achieves significant improvements in social reasoning capabilities.
  arXiv  Detail & Related papers  (2026-01-18T09:10:08Z)
• SoulSeek: Exploring the Use of Social Cues in LLM-based Information Seeking [23.78415242490134]
  Social cues play a crucial role in human information seeking by helping individuals judge relevance and trustworthiness.<n>Existing LLM-based search systems rely on semantic features, creating a misalignment with the socialized cognition underlying natural information seeking.<n>We propose design implications emphasizing better social-knowledge understanding, personalized cue settings, and controllable interactions.
  arXiv  Detail & Related papers  (2026-01-03T07:09:10Z)
• Social Simulations with Large Language Model Risk Utopian Illusion [61.358959720048354]
  We introduce a systematic framework for analyzing large language models' behavior in social simulation.<n>Our approach simulates multi-agent interactions through chatroom-style conversations and analyzes them across five linguistic dimensions.<n>Our findings reveal that LLMs do not faithfully reproduce genuine human behavior but instead reflect overly idealized versions of it.
  arXiv  Detail & Related papers  (2025-10-24T06:08:41Z)
• Social Welfare Function Leaderboard: When LLM Agents Allocate Social Welfare [87.06241096619112]
  Large language models (LLMs) are increasingly entrusted with high-stakes decisions that affect human welfare.<n>We introduce the Social Welfare Function Benchmark, a dynamic simulation environment where an LLM acts as a sovereign allocator.<n>We evaluate 20 state-of-the-art LLMs and present the first leaderboard for social welfare allocation.
  arXiv  Detail & Related papers  (2025-10-01T17:52:31Z)
• SCRAG: Social Computing-Based Retrieval Augmented Generation for Community Response Forecasting in Social Media Environments [8.743208265682014]
  SCRAG is a prediction framework inspired by social computing.<n>It forecast community responses to real or hypothetical social media posts.<n>It can be used by public relations specialists to craft messaging in ways that avoid unintended misinterpretations.
  arXiv  Detail & Related papers  (2025-04-18T15:02:31Z)
• Can LLMs Simulate Social Media Engagement? A Study on Action-Guided Response Generation [51.44040615856536]
  This paper analyzes large language models' ability to simulate social media engagement through action guided response generation.<n>We benchmark GPT-4o-mini, O1-mini, and DeepSeek-R1 in social media engagement simulation regarding a major societal event.
  arXiv  Detail & Related papers  (2025-02-17T17:43:08Z)
• Persuasion with Large Language Models: a Survey [49.86930318312291]
  Large Language Models (LLMs) have created new disruptive possibilities for persuasive communication. In areas such as politics, marketing, public health, e-commerce, and charitable giving, such LLM Systems have already achieved human-level or even super-human persuasiveness. Our survey suggests that the current and future potential of LLM-based persuasion poses profound ethical and societal risks.
  arXiv  Detail & Related papers  (2024-11-11T10:05:52Z)
• Social Debiasing for Fair Multi-modal LLMs [59.61512883471714]
  Multi-modal Large Language Models (MLLMs) have dramatically advanced the research field and delivered powerful vision-language understanding capabilities.<n>These models often inherit deep-rooted social biases from their training data, leading to uncomfortable responses with respect to attributes such as race and gender.<n>This paper addresses the issue of social biases in MLLMs by introducing a comprehensive counterfactual dataset with multiple social concepts.
  arXiv  Detail & Related papers  (2024-08-13T02:08:32Z)
• Prompting Fairness: Integrating Causality to Debias Large Language Models [19.76215433424235]
  Large language models (LLMs) are susceptible to generating biased and discriminatory responses.<n>We propose a causality-guided debiasing framework to tackle social biases.
  arXiv  Detail & Related papers  (2024-03-13T17:46:28Z)
• SOTOPIA-$π$: Interactive Learning of Socially Intelligent Language Agents [73.35393511272791]
  We propose an interactive learning method, SOTOPIA-$pi$, improving the social intelligence of language agents. This method leverages behavior cloning and self-reinforcement training on filtered social interaction data according to large language model (LLM) ratings.
  arXiv  Detail & Related papers  (2024-03-13T17:17:48Z)
• Do LLM Agents Exhibit Social Behavior? [5.094340963261968]
  State-Understanding-Value-Action (SUVA) is a framework to systematically analyze responses in social contexts. It assesses social behavior through both their final decisions and the response generation processes leading to those decisions. We demonstrate that utterance-based reasoning reliably predicts LLMs' final actions.
  arXiv  Detail & Related papers  (2023-12-23T08:46:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.