Post-hoc Study of Climate Microtargeting on Social Media Ads with LLMs: Thematic Insights and Fairness Evaluation

• URL: http://arxiv.org/abs/2410.05401v4
• Date: Tue, 23 Sep 2025 01:45:34 GMT
• Title: Post-hoc Study of Climate Microtargeting on Social Media Ads with LLMs: Thematic Insights and Fairness Evaluation
• Abstract summary: This study presents a post-hoc analysis of microtargeting practices within climate campaigns.<n>We evaluate the ability of large language models (LLMs) to accurately predict the intended demographic targets.<n>We conduct a comprehensive fairness analysis to uncover biases in model predictions.
• Score: 24.599293872173362
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Climate change communication on social media increasingly employs microtargeting strategies to effectively reach and influence specific demographic groups. This study presents a post-hoc analysis of microtargeting practices within climate campaigns by leveraging large language models (LLMs) to examine Meta (previously known as Facebook) advertisements. Our analysis focuses on two key aspects: demographic targeting and fairness. We evaluate the ability of LLMs to accurately predict the intended demographic targets, such as gender and age group. Furthermore, we instruct the LLMs to generate explanations for their classifications, providing transparent reasoning behind each decision. These explanations reveal the specific thematic elements used to engage different demographic segments, highlighting distinct strategies tailored to various audiences. Our findings show that young adults are primarily targeted through messages emphasizing activism and environmental consciousness, while women are engaged through themes related to caregiving roles and social advocacy. Additionally, we conduct a comprehensive fairness analysis to uncover biases in model predictions. We assess disparities in accuracy and error rates across demographic groups using established fairness metrics such as Demographic Parity, Equal Opportunity, and Predictive Equality. Our findings indicate that while LLMs perform well overall, certain biases exist, particularly in the classification of male audiences. The analysis of thematic explanations uncovers recurring patterns in messaging strategies tailored to various demographic groups, while the fairness analysis underscores the need for more inclusive targeting methods. This study provides a valuable framework for future research aimed at enhancing transparency, accountability, and inclusivity in social media-driven climate campaigns.

Related papers

• Sociodemographic Biases in Educational Counselling by Large Language Models [61.30199414932204]
  This study examines sociodemographic biases in Large Language Models (LLMs)<n>We evaluate responses from six LLMs answering questions about 900 vignettes describing students in diverse circumstances.
  arXiv  Detail & Related papers  (2026-04-03T14:08:22Z)
• Who Gets Which Message? Auditing Demographic Bias in LLM-Generated Targeted Text [8.06425428468097]
  Large language models (LLMs) are increasingly capable of generating personalized, persuasive text at scale.<n>This paper presents the first systematic analysis of how LLMs behave when tasked with demographic-conditioned targeted messaging.
  arXiv  Detail & Related papers  (2026-01-23T21:07:59Z)
• Mitigating Bias with Words: Inducing Demographic Ambiguity in Face Recognition Templates by Text Encoding [18.08946802592489]
  Face recognition systems are often prone to demographic biases, partially due to the entanglement of demographic-specific information with identity-relevant features in facial embeddings.<n>We propose a novel strategy, Unified Text-Image Embedding (UTIE), which aims to induce demographic ambiguity in face embeddings.<n>We show that UTIE consistently reduces bias metrics while maintaining, or even improving in several cases, the face verification accuracy.
  arXiv  Detail & Related papers  (2025-12-05T10:41:58Z)
• Evaluating LLMs for Demographic-Targeted Social Bias Detection: A Comprehensive Benchmark Study [1.6682715542079583]
  Large-scale web-scraped text corpora used to train general-purpose AI models often contain harmful demographic-targeted social biases.<n>We present a comprehensive evaluation framework aimed at English texts to assess the ability of LLMs in detecting demographic-targeted social biases.<n>We then conduct a systematic evaluation with models across scales and techniques, including prompting, in-context learning, and fine-tuning.
  arXiv  Detail & Related papers  (2025-10-06T09:45:32Z)
• The Prompt Makes the Person(a): A Systematic Evaluation of Sociodemographic Persona Prompting for Large Language Models [3.2919397230854983]
  We show how different persona prompt strategies, specifically role adoption formats and demographic priming strategies, influence large language models.<n>Our findings show that LLMs struggle to simulate marginalized groups, particularly nonbinary, Hispanic, and Middle Eastern identities.<n>Specifically, we find that prompting in an interview-style format and name-based priming can help reduce stereotyping and improve alignment.
  arXiv  Detail & Related papers  (2025-07-21T21:23:29Z)
• Persuasion at Play: Understanding Misinformation Dynamics in Demographic-Aware Human-LLM Interactions [27.38030183605309]
  Large language models (LLMs) generate persuasive content at scale and reinforce existing biases. This study investigates the bidirectional persuasion dynamics between LLMs and humans when exposed to misinformative content. Our findings show that demographic factors influence susceptibility to misinformation in LLMs, closely reflecting the demographic-based patterns seen in human susceptibility.
  arXiv  Detail & Related papers  (2025-03-03T20:30:22Z)
• Mind the (Belief) Gap: Group Identity in the World of LLMs [22.96432452893247]
  Social biases and belief-driven behaviors can significantly impact Large Language Models (LLMs) decisions on several tasks. We present a multi-agent framework that simulates belief congruence, a classical group psychology theory that plays a crucial role in shaping societal interactions and preferences.
  arXiv  Detail & Related papers  (2025-03-03T19:50:52Z)
• Unmasking Implicit Bias: Evaluating Persona-Prompted LLM Responses in Power-Disparate Social Scenarios [4.626073646852022]
  We introduce a novel framework using cosine distance to measure semantic shifts in responses. We assess how demographic prompts affect response quality across power-disparate social scenarios. Our findings suggest a "default persona" bias toward middle-aged, able-bodied, native-born, Caucasian, atheistic males with centrist views.
  arXiv  Detail & Related papers  (2025-03-03T13:44:03Z)
• NewsInterview: a Dataset and a Playground to Evaluate LLMs' Ground Gap via Informational Interviews [65.35458530702442]
  We focus on journalistic interviews, a domain rich in grounding communication and abundant in data. We curate a dataset of 40,000 two-person informational interviews from NPR and CNN. LLMs are significantly less likely than human interviewers to use acknowledgements and to pivot to higher-level questions.
  arXiv  Detail & Related papers  (2024-11-21T01:37:38Z)
• Persuasion with Large Language Models: a Survey [49.86930318312291]
  Large Language Models (LLMs) have created new disruptive possibilities for persuasive communication. In areas such as politics, marketing, public health, e-commerce, and charitable giving, such LLM Systems have already achieved human-level or even super-human persuasiveness. Our survey suggests that the current and future potential of LLM-based persuasion poses profound ethical and societal risks.
  arXiv  Detail & Related papers  (2024-11-11T10:05:52Z)
• Hate Personified: Investigating the role of LLMs in content moderation [64.26243779985393]
  For subjective tasks such as hate detection, where people perceive hate differently, the Large Language Model's (LLM) ability to represent diverse groups is unclear. By including additional context in prompts, we analyze LLM's sensitivity to geographical priming, persona attributes, and numerical information to assess how well the needs of various groups are reflected.
  arXiv  Detail & Related papers  (2024-10-03T16:43:17Z)
• Evaluating Implicit Bias in Large Language Models by Attacking From a Psychometric Perspective [66.34066553400108]
  We conduct a rigorous evaluation of large language models' implicit bias towards certain demographics. Inspired by psychometric principles, we propose three attack approaches, i.e., Disguise, Deception, and Teaching. Our methods can elicit LLMs' inner bias more effectively than competitive baselines.
  arXiv  Detail & Related papers  (2024-06-20T06:42:08Z)
• Advancing Annotation of Stance in Social Media Posts: A Comparative Analysis of Large Language Models and Crowd Sourcing [2.936331223824117]
  Large Language Models (LLMs) for automated text annotation in social media posts has garnered significant interest. We analyze the performance of eight open-source and proprietary LLMs for annotating the stance expressed in social media posts. A significant finding of our study is that the explicitness of text expressing a stance plays a critical role in how faithfully LLMs' stance judgments match humans'
  arXiv  Detail & Related papers  (2024-06-11T17:26:07Z)
• Evaluating Interventional Reasoning Capabilities of Large Language Models [58.52919374786108]
  Large language models (LLMs) can estimate causal effects under interventions on different parts of a system. We conduct empirical analyses to evaluate whether LLMs can accurately update their knowledge of a data-generating process in response to an intervention. We create benchmarks that span diverse causal graphs (e.g., confounding, mediation) and variable types, and enable a study of intervention-based reasoning.
  arXiv  Detail & Related papers  (2024-04-08T14:15:56Z)
• Explaining Large Language Models Decisions Using Shapley Values [1.223779595809275]
  Large language models (LLMs) have opened up exciting possibilities for simulating human behavior and cognitive processes. However, the validity of utilizing LLMs as stand-ins for human subjects remains uncertain. This paper presents a novel approach based on Shapley values to interpret LLM behavior and quantify the relative contribution of each prompt component to the model's output.
  arXiv  Detail & Related papers  (2024-03-29T22:49:43Z)
• Comprehensive Reassessment of Large-Scale Evaluation Outcomes in LLMs: A Multifaceted Statistical Approach [64.42462708687921]
  Evaluations have revealed that factors such as scaling, training types, architectures and other factors profoundly impact the performance of LLMs. Our study embarks on a thorough re-examination of these LLMs, targeting the inadequacies in current evaluation methods. This includes the application of ANOVA, Tukey HSD tests, GAMM, and clustering technique.
  arXiv  Detail & Related papers  (2024-03-22T14:47:35Z)
• Discovering Latent Themes in Social Media Messaging: A Machine-in-the-Loop Approach Integrating LLMs [22.976609127865732]
  We introduce a novel approach to uncovering latent themes in social media messaging. Our work sheds light on the dynamic nature of social media, revealing the shifts in the thematic focus of messaging in response to real-world events.
  arXiv  Detail & Related papers  (2024-03-15T21:54:00Z)
• A Group Fairness Lens for Large Language Models [34.0579082699443]
  Large language models can perpetuate biases and unfairness when deployed in social media contexts. We propose evaluating LLM biases from a group fairness lens using a novel hierarchical schema characterizing diverse social groups. We pioneer a novel chain-of-thought method GF-Think to mitigate biases of LLMs from a group fairness perspective.
  arXiv  Detail & Related papers  (2023-12-24T13:25:15Z)
• Sociodemographic Prompting is Not Yet an Effective Approach for Simulating Subjective Judgments with LLMs [13.744746481528711]
  Large Language Models (LLMs) are widely used to simulate human responses across diverse contexts. We evaluate nine popular LLMs on their ability to understand demographic differences in two subjective judgment tasks: politeness and offensiveness. We find that in zero-shot settings, most models' predictions for both tasks align more closely with labels from White participants than those from Asian or Black participants.
  arXiv  Detail & Related papers  (2023-11-16T10:02:24Z)
• On the steerability of large language models toward data-driven personas [98.9138902560793]
  Large language models (LLMs) are known to generate biased responses where the opinions of certain groups and populations are underrepresented. Here, we present a novel approach to achieve controllable generation of specific viewpoints using LLMs.
  arXiv  Detail & Related papers  (2023-11-08T19:01:13Z)
• Bias and Fairness in Large Language Models: A Survey [73.87651986156006]
  We present a comprehensive survey of bias evaluation and mitigation techniques for large language models (LLMs) We first consolidate, formalize, and expand notions of social bias and fairness in natural language processing. We then unify the literature by proposing three intuitive, two for bias evaluation, and one for mitigation.
  arXiv  Detail & Related papers  (2023-09-02T00:32:55Z)
• Measuring Fairness Under Unawareness of Sensitive Attributes: A Quantification-Based Approach [131.20444904674494]
  We tackle the problem of measuring group fairness under unawareness of sensitive attributes. We show that quantification approaches are particularly suited to tackle the fairness-under-unawareness problem.
  arXiv  Detail & Related papers  (2021-09-17T13:45:46Z)
• Mitigating Face Recognition Bias via Group Adaptive Classifier [53.15616844833305]
  This work aims to learn a fair face representation, where faces of every group could be more equally represented. Our work is able to mitigate face recognition bias across demographic groups while maintaining the competitive accuracy.
  arXiv  Detail & Related papers  (2020-06-13T06:43:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.