Language Models Trained on Media Diets Can Predict Public Opinion
- URL: http://arxiv.org/abs/2303.16779v1
- Date: Tue, 28 Mar 2023 06:08:25 GMT
- Title: Language Models Trained on Media Diets Can Predict Public Opinion
- Authors: Eric Chu, Jacob Andreas, Stephen Ansolabehere, Deb Roy
- Abstract summary: We introduce a novel approach to probe media diet models that emulate the opinions of subpopulations that have consumed a set of media.
Our studies indicate that this approach is (1) predictive of human judgements found in survey response distributions, (2) more accurate at modeling people who follow media more closely, and (3) aligned with literature on which types of opinions are affected by media consumption.
- Score: 43.824336518942
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Public opinion reflects and shapes societal behavior, but the traditional
survey-based tools to measure it are limited. We introduce a novel approach to
probe media diet models -- language models adapted to online news, TV
broadcast, or radio show content -- that can emulate the opinions of
subpopulations that have consumed a set of media. To validate this method, we
use as ground truth the opinions expressed in U.S. nationally representative
surveys on COVID-19 and consumer confidence. Our studies indicate that this
approach is (1) predictive of human judgements found in survey response
distributions and robust to phrasing and channels of media exposure, (2) more
accurate at modeling people who follow media more closely, and (3) aligned with
literature on which types of opinions are affected by media consumption.
Probing language models provides a powerful new method for investigating media
effects, has practical applications in supplementing polls and forecasting
public opinion, and suggests a need for further study of the surprising
fidelity with which neural language models can predict human responses.
Related papers
- AMMeBa: A Large-Scale Survey and Dataset of Media-Based Misinformation In-The-Wild [1.4193873432298625]
We show the results of a two-year study using human raters to annotate online media-based misinformation.
We show the rise of generative AI-based content in misinformation claims.
We also show "simple" methods dominated historically, particularly context manipulations.
arXiv Detail & Related papers (2024-05-19T23:05:53Z) - Quantifying Media Influence on Covid-19 Mask-Wearing Beliefs [0.8192907805418583]
This study contributes a dataset of U.S. news media stories, annotated according to Howard 2020's Face Mask Perception Scale for their statements regarding Covid-19 mask-wearing.
We demonstrate fine-grained correlations between media messaging and empirical opinion polling data from a Gallup survey conducted during the same period.
We also demonstrate that the data can be used for quantitative analysis of pro- and anti-mask sentiment throughout the period.
arXiv Detail & Related papers (2024-03-06T13:09:40Z) - Random Silicon Sampling: Simulating Human Sub-Population Opinion Using a
Large Language Model Based on Group-Level Demographic Information [15.435605802794408]
Large language models exhibit societal biases associated with demographic information.
We propose "random silicon sampling," a method to emulate the opinions of the human population sub-group.
We find that language models can generate response distributions remarkably similar to the actual U.S. public opinion polls.
arXiv Detail & Related papers (2024-02-28T08:09:14Z) - Bias in Opinion Summarisation from Pre-training to Adaptation: A Case
Study in Political Bias [4.964212137957899]
Opinion summarisation aims to summarise the salient information and opinions presented in documents such as product reviews, discussion forums, and social media texts.
generating biased summaries has the risk of potentially swaying public opinion.
arXiv Detail & Related papers (2024-02-01T04:15:59Z) - Decoding Susceptibility: Modeling Misbelief to Misinformation Through a Computational Approach [61.04606493712002]
Susceptibility to misinformation describes the degree of belief in unverifiable claims that is not observable.
Existing susceptibility studies heavily rely on self-reported beliefs.
We propose a computational approach to model users' latent susceptibility levels.
arXiv Detail & Related papers (2023-11-16T07:22:56Z) - Decoding the Silent Majority: Inducing Belief Augmented Social Graph
with Large Language Model for Response Forecasting [74.68371461260946]
SocialSense is a framework that induces a belief-centered graph on top of an existent social network, along with graph-based propagation to capture social dynamics.
Our method surpasses existing state-of-the-art in experimental evaluations for both zero-shot and supervised settings.
arXiv Detail & Related papers (2023-10-20T06:17:02Z) - Measuring the Effect of Influential Messages on Varying Personas [67.1149173905004]
We present a new task, Response Forecasting on Personas for News Media, to estimate the response a persona might have upon seeing a news message.
The proposed task not only introduces personalization in the modeling but also predicts the sentiment polarity and intensity of each response.
This enables more accurate and comprehensive inference on the mental state of the persona.
arXiv Detail & Related papers (2023-05-25T21:01:00Z) - Bridging the Gap: A Survey on Integrating (Human) Feedback for Natural
Language Generation [68.9440575276396]
This survey aims to provide an overview of the recent research that has leveraged human feedback to improve natural language generation.
First, we introduce an encompassing formalization of feedback, and identify and organize existing research into a taxonomy following this formalization.
Second, we discuss how feedback can be described by its format and objective, and cover the two approaches proposed to use feedback (either for training or decoding): directly using the feedback or training feedback models.
Third, we provide an overview of the nascent field of AI feedback, which exploits large language models to make judgments based on a set of principles and minimize the need for
arXiv Detail & Related papers (2023-05-01T17:36:06Z) - Unveiling the Hidden Agenda: Biases in News Reporting and Consumption [59.55900146668931]
We build a six-year dataset on the Italian vaccine debate and adopt a Bayesian latent space model to identify narrative and selection biases.
We found a nonlinear relationship between biases and engagement, with higher engagement for extreme positions.
Analysis of news consumption on Twitter reveals common audiences among news outlets with similar ideological positions.
arXiv Detail & Related papers (2023-01-14T18:58:42Z) - GREENER: Graph Neural Networks for News Media Profiling [24.675574340841163]
We study the problem of profiling news media on the Web with respect to their factuality of reporting and bias.
Our main focus is on modeling the similarity between media outlets based on the overlap of their audience.
Prediction accuracy is found to improve by 2.5-27 macro-F1 points for the two tasks.
arXiv Detail & Related papers (2022-11-10T12:46:29Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.