A Large-Scale Simulation on Large Language Models for Decision-Making in Political Science

• URL: http://arxiv.org/abs/2412.15291v4
• Date: Thu, 10 Apr 2025 02:50:31 GMT
• Title: A Large-Scale Simulation on Large Language Models for Decision-Making in Political Science
• Authors: Chenxiao Yu, Jinyi Ye, Yuangang Li, Zheng Li, Emilio Ferrara, Xiyang Hu, Yue Zhao,
• Abstract summary: We develop a theory-driven, multi-step reasoning framework to simulate voter decision-making at scale.<n>We conduct large-scale simulations of recent U.S. presidential elections using synthetic personas calibrated to real-world voter data.
• Score: 18.521101885334673
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While LLMs have demonstrated remarkable capabilities in text generation and reasoning, their ability to simulate human decision-making -- particularly in political contexts -- remains an open question. However, modeling voter behavior presents unique challenges due to limited voter-level data, evolving political landscapes, and the complexity of human reasoning. In this study, we develop a theory-driven, multi-step reasoning framework that integrates demographic, temporal and ideological factors to simulate voter decision-making at scale. Using synthetic personas calibrated to real-world voter data, we conduct large-scale simulations of recent U.S. presidential elections. Our method significantly improves simulation accuracy while mitigating model biases. We examine its robustness by comparing performance across different LLMs. We further investigate the challenges and constraints that arise from LLM-based political simulations. Our work provides both a scalable framework for modeling political decision-making behavior and insights into the promise and limitations of using LLMs in political science research.

Related papers

• An Overview of Large Language Models for Statisticians [109.38601458831545]
  Large Language Models (LLMs) have emerged as transformative tools in artificial intelligence (AI) This paper explores potential areas where statisticians can make important contributions to the development of LLMs. We focus on issues such as uncertainty quantification, interpretability, fairness, privacy, watermarking and model adaptation.
  arXiv  Detail & Related papers  (2025-02-25T03:40:36Z)
• Human Preferences in Large Language Model Latent Space: A Technical Analysis on the Reliability of Synthetic Data in Voting Outcome Prediction [5.774786149181393]
  We analyze how demographic attributes and prompt variations influence latent opinion mappings in large language models (LLMs) We find that LLM-generated data fails to replicate the variance observed in real-world human responses. In the political space, persona-to-party mappings exhibit limited differentiation, resulting in synthetic data that lacks the nuanced distribution of opinions found in survey data.
  arXiv  Detail & Related papers  (2025-02-22T16:25:33Z)
• Political-LLM: Large Language Models in Political Science [159.95299889946637]
  Large language models (LLMs) have been widely adopted in political science tasks.<n>Political-LLM aims to advance the comprehensive understanding of integrating LLMs into computational political science.
  arXiv  Detail & Related papers  (2024-12-09T08:47:50Z)
• ElectionSim: Massive Population Election Simulation Powered by Large Language Model Driven Agents [70.17229548653852]
  We introduce ElectionSim, an innovative election simulation framework based on large language models. We present a million-level voter pool sampled from social media platforms to support accurate individual simulation. We also introduce PPE, a poll-based presidential election benchmark to assess the performance of our framework under the U.S. presidential election scenario.
  arXiv  Detail & Related papers  (2024-10-28T05:25:50Z)
• Large Language Models Reflect the Ideology of their Creators [71.65505524599888]
  Large language models (LLMs) are trained on vast amounts of data to generate natural language. This paper shows that the ideological stance of an LLM appears to reflect the worldview of its creators.
  arXiv  Detail & Related papers  (2024-10-24T04:02:30Z)
• Towards More Accurate US Presidential Election via Multi-step Reasoning with Large Language Models [12.582222782098587]
  Election prediction poses unique challenges, such as limited voter-level data, rapidly changing political landscapes, and the need to model complex human behavior.<n>We introduce a multi-step reasoning framework designed for political analysis.<n>Our approach is validated on real-world data from the American National Election Studies (ANES) 2016 and 2020.
  arXiv  Detail & Related papers  (2024-10-21T06:18:53Z)
• Designing Domain-Specific Large Language Models: The Critical Role of Fine-Tuning in Public Opinion Simulation [0.0]
  This paper introduces a novel fine-tuning approach that integrates socio-demographic data from the UK Household Longitudinal Study. By emulating diverse synthetic profiles, the fine-tuned models significantly outperform pre-trained counterparts. Its broader implications include deploying LLMs in domains like healthcare and education, fostering inclusive and data-driven decision-making.
  arXiv  Detail & Related papers  (2024-09-28T10:39:23Z)
• United in Diversity? Contextual Biases in LLM-Based Predictions of the 2024 European Parliament Elections [45.84205238554709]
  Large language models (LLMs) are perceived by some as having the potential to revolutionize social science research. In this study, we examine to what extent LLM-based predictions of public opinion exhibit context-dependent biases. We predict voting behavior in the 2024 European Parliament elections using a state-of-the-art LLM.
  arXiv  Detail & Related papers  (2024-08-29T16:01:06Z)
• Representation Bias in Political Sample Simulations with Large Language Models [54.48283690603358]
  This study seeks to identify and quantify biases in simulating political samples with Large Language Models. Using the GPT-3.5-Turbo model, we leverage data from the American National Election Studies, German Longitudinal Election Study, Zuobiao dataset, and China Family Panel Studies.
  arXiv  Detail & Related papers  (2024-07-16T05:52:26Z)
• Can LLMs Help Predict Elections? (Counter)Evidence from the World's Largest Democracy [3.0915192911449796]
  The study of how social media affects the formation of public opinion and its influence on political results has been a popular field of inquiry. We introduce a new method: harnessing the capabilities of Large Language Models (LLMs) to examine social media data and forecast election outcomes.
  arXiv  Detail & Related papers  (2024-05-13T15:13:23Z)
• Large Language Models (LLMs) as Agents for Augmented Democracy [6.491009626125319]
  We explore an augmented democracy system built on off-the-shelf LLMs fine-tuned to augment data on citizen's preferences. We use a train-test cross-validation setup to estimate the accuracy with which the LLMs predict both: a subject's individual political choices and the aggregate preferences of the full sample of participants.
  arXiv  Detail & Related papers  (2024-05-06T13:23:57Z)
• Whose Side Are You On? Investigating the Political Stance of Large Language Models [56.883423489203786]
  We investigate the political orientation of Large Language Models (LLMs) across a spectrum of eight polarizing topics. Our investigation delves into the political alignment of LLMs across a spectrum of eight polarizing topics, spanning from abortion to LGBTQ issues. The findings suggest that users should be mindful when crafting queries, and exercise caution in selecting neutral prompt language.
  arXiv  Detail & Related papers  (2024-03-15T04:02:24Z)
• LLM-driven Imitation of Subrational Behavior : Illusion or Reality? [3.2365468114603937]
  Existing work highlights the ability of Large Language Models to address complex reasoning tasks and mimic human communication. We propose to investigate the use of LLMs to generate synthetic human demonstrations, which are then used to learn subrational agent policies. We experimentally evaluate the ability of our framework to model sub-rationality through four simple scenarios.
  arXiv  Detail & Related papers  (2024-02-13T19:46:39Z)
• CoMPosT: Characterizing and Evaluating Caricature in LLM Simulations [61.9212914612875]
  We present a framework to characterize LLM simulations using four dimensions: Context, Model, Persona, and Topic. We use this framework to measure open-ended LLM simulations' susceptibility to caricature, defined via two criteria: individuation and exaggeration. We find that for GPT-4, simulations of certain demographics (political and marginalized groups) and topics (general, uncontroversial) are highly susceptible to caricature.
  arXiv  Detail & Related papers  (2023-10-17T18:00:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.