Large-Scale, Longitudinal Study of Large Language Models During the 2024 US Election Season

• URL: http://arxiv.org/abs/2509.18446v1
• Date: Mon, 22 Sep 2025 22:04:19 GMT
• Title: Large-Scale, Longitudinal Study of Large Language Models During the 2024 US Election Season
• Authors: Sarah H. Cen, Andrew Ilyas, Hedi Driss, Charlotte Park, Aspen Hopkins, Chara Podimata, Aleksander Mądry,
• Abstract summary: The 2024 US presidential election is the first major contest to occur in the US since the popularization of large language models (LLMs)<n>This moment raises urgent questions about how LLMs may shape the information ecosystem and influence political discourse.<n>We conduct a large-scale, longitudinal study of 12 models, queried using a structured survey with over 12,000 questions on a near-daily cadence from July through November 2024.
• Score: 43.092041950140164
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The 2024 US presidential election is the first major contest to occur in the US since the popularization of large language models (LLMs). Building on lessons from earlier shifts in media (most notably social media's well studied role in targeted messaging and political polarization) this moment raises urgent questions about how LLMs may shape the information ecosystem and influence political discourse. While platforms have announced some election safeguards, how well they work in practice remains unclear. Against this backdrop, we conduct a large-scale, longitudinal study of 12 models, queried using a structured survey with over 12,000 questions on a near-daily cadence from July through November 2024. Our design systematically varies content and format, resulting in a rich dataset that enables analyses of the models' behavior over time (e.g., across model updates), sensitivity to steering, responsiveness to instructions, and election-related knowledge and "beliefs." In the latter half of our work, we perform four analyses of the dataset that (i) study the longitudinal variation of model behavior during election season, (ii) illustrate the sensitivity of election-related responses to demographic steering, (iii) interrogate the models' beliefs about candidates' attributes, and (iv) reveal the models' implicit predictions of the election outcome. To facilitate future evaluations of LLMs in electoral contexts, we detail our methodology, from question generation to the querying pipeline and third-party tooling. We also publicly release our dataset at https://huggingface.co/datasets/sarahcen/llm-election-data-2024

Related papers

• Do LLMs Track Public Opinion? A Multi-Model Study of Favorability Predictions in the 2024 U.S. Presidential Election [5.071155887901222]
  We investigate whether Large Language Models (LLMs) can track public opinion as measured by exit polls during the 2024 U.S. presidential election cycle.<n>We evaluate predictions from nine LLM configurations against a curated set of five high-quality polls from major organizations including Reuters, CNN, Gallup, Quinnipiac, and ABC.<n>We conclude that off-the-shelf LLMs do not reliably track polls when queried in a straightforward manner and discuss implications for election forecasting.
  arXiv  Detail & Related papers  (2026-02-06T01:52:13Z)
• From Keywords to Clusters: AI-Driven Analysis of YouTube Comments to Reveal Election Issue Salience in 2024 [1.521610318673192]
  Immigration and democracy were the most frequently and consistently invoked issues in user comments on the analyzed YouTube videos.<n>These results corroborate certain findings of post-election surveys but also refute the supposed importance of inflation as an election issue.
  arXiv  Detail & Related papers  (2025-10-09T06:02:10Z)
• A Large-Scale Simulation on Large Language Models for Decision-Making in Political Science [18.521101885334673]
  We develop a theory-driven, multi-step reasoning framework to simulate voter decision-making at scale.<n>We conduct large-scale simulations of recent U.S. presidential elections using synthetic personas calibrated to real-world voter data.
  arXiv  Detail & Related papers  (2024-12-19T07:10:51Z)
• ElectionSim: Massive Population Election Simulation Powered by Large Language Model Driven Agents [70.17229548653852]
  We introduce ElectionSim, an innovative election simulation framework based on large language models. We present a million-level voter pool sampled from social media platforms to support accurate individual simulation. We also introduce PPE, a poll-based presidential election benchmark to assess the performance of our framework under the U.S. presidential election scenario.
  arXiv  Detail & Related papers  (2024-10-28T05:25:50Z)
• Towards More Accurate US Presidential Election via Multi-step Reasoning with Large Language Models [12.582222782098587]
  Election prediction poses unique challenges, such as limited voter-level data, rapidly changing political landscapes, and the need to model complex human behavior.<n>We introduce a multi-step reasoning framework designed for political analysis.<n>Our approach is validated on real-world data from the American National Election Studies (ANES) 2016 and 2020.
  arXiv  Detail & Related papers  (2024-10-21T06:18:53Z)
• Representation Bias in Political Sample Simulations with Large Language Models [54.48283690603358]
  This study seeks to identify and quantify biases in simulating political samples with Large Language Models. Using the GPT-3.5-Turbo model, we leverage data from the American National Election Studies, German Longitudinal Election Study, Zuobiao dataset, and China Family Panel Studies.
  arXiv  Detail & Related papers  (2024-07-16T05:52:26Z)
• Investigating LLMs as Voting Assistants via Contextual Augmentation: A Case Study on the European Parliament Elections 2024 [22.471701390730185]
  In light of the recent 2024 European Parliament elections, we are investigating if LLMs can be used as Voting Advice Applications (VAAs) We evaluate MISTRAL and MIXTRAL models and evaluate their accuracy in predicting the stance of political parties based on the latest "EU and I" voting assistance questionnaire. We find that MIXTRAL is highly accurate with an 82% accuracy on average with a significant performance disparity across different political groups.
  arXiv  Detail & Related papers  (2024-07-11T13:29:28Z)
• Can LLMs Help Predict Elections? (Counter)Evidence from the World's Largest Democracy [3.0915192911449796]
  The study of how social media affects the formation of public opinion and its influence on political results has been a popular field of inquiry. We introduce a new method: harnessing the capabilities of Large Language Models (LLMs) to examine social media data and forecast election outcomes.
  arXiv  Detail & Related papers  (2024-05-13T15:13:23Z)
• Whose Side Are You On? Investigating the Political Stance of Large Language Models [56.883423489203786]
  We investigate the political orientation of Large Language Models (LLMs) across a spectrum of eight polarizing topics. Our investigation delves into the political alignment of LLMs across a spectrum of eight polarizing topics, spanning from abortion to LGBTQ issues. The findings suggest that users should be mindful when crafting queries, and exercise caution in selecting neutral prompt language.
  arXiv  Detail & Related papers  (2024-03-15T04:02:24Z)
• Design and analysis of tweet-based election models for the 2021 Mexican legislative election [55.41644538483948]
  We use a dataset of 15 million election-related tweets in the six months preceding election day. We find that models using data with geographical attributes determine the results of the election with better precision and accuracy than conventional polling methods.
  arXiv  Detail & Related papers  (2023-01-02T12:40:05Z)
• Electoral Forecasting Using a Novel Temporal Attenuation Model: Predicting the US Presidential Elections [91.3755431537592]
  We develop a novel macro-scale temporal attenuation (TA) model, which uses pre-election poll data to improve forecasting accuracy. Our hypothesis is that the timing of publicizing opinion polls plays a significant role in how opinion oscillates, especially right before elections. We present two different implementations of the TA model, which accumulate an average forecasting error of 2.8-3.28 points over the 48-year period.
  arXiv  Detail & Related papers  (2020-04-30T09:21:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.