The Self-Perception and Political Biases of ChatGPT
- URL: http://arxiv.org/abs/2304.07333v1
- Date: Fri, 14 Apr 2023 18:06:13 GMT
- Title: The Self-Perception and Political Biases of ChatGPT
- Authors: J\'er\^ome Rutinowski, Sven Franke, Jan Endendyk, Ina Dormuth, Markus
Pauly
- Abstract summary: This contribution analyzes the self-perception and political biases of OpenAI's Large Language Model ChatGPT.
The political compass test revealed a bias towards progressive and libertarian views.
Political questionnaires for the G7 member states indicated a bias towards progressive views but no significant bias between authoritarian and libertarian views.
- Score: 0.0
- License: http://creativecommons.org/licenses/by-nc-nd/4.0/
- Abstract: This contribution analyzes the self-perception and political biases of
OpenAI's Large Language Model ChatGPT. Taking into account the first
small-scale reports and studies that have emerged, claiming that ChatGPT is
politically biased towards progressive and libertarian points of view, this
contribution aims to provide further clarity on this subject. For this purpose,
ChatGPT was asked to answer the questions posed by the political compass test
as well as similar questionnaires that are specific to the respective politics
of the G7 member states. These eight tests were repeated ten times each and
revealed that ChatGPT seems to hold a bias towards progressive views. The
political compass test revealed a bias towards progressive and libertarian
views, with the average coordinates on the political compass being (-6.48,
-5.99) (with (0, 0) the center of the compass, i.e., centrism and the axes
ranging from -10 to 10), supporting the claims of prior research. The political
questionnaires for the G7 member states indicated a bias towards progressive
views but no significant bias between authoritarian and libertarian views,
contradicting the findings of prior reports, with the average coordinates being
(-3.27, 0.58). In addition, ChatGPT's Big Five personality traits were tested
using the OCEAN test and its personality type was queried using the
Myers-Briggs Type Indicator (MBTI) test. Finally, the maliciousness of ChatGPT
was evaluated using the Dark Factor test. These three tests were also repeated
ten times each, revealing that ChatGPT perceives itself as highly open and
agreeable, has the Myers-Briggs personality type ENFJ, and is among the 15% of
test-takers with the least pronounced dark traits.
Related papers
- Quite Good, but Not Enough: Nationality Bias in Large Language Models -- A Case Study of ChatGPT [4.998396762666333]
This study investigates nationality bias in ChatGPT (GPT-3.5), a large language model (LLM) designed for text generation.
The research covers 195 countries, 4 temperature settings, and 3 distinct prompt types, generating 4,680 discourses about nationality descriptions in Chinese and English.
arXiv Detail & Related papers (2024-05-11T12:11:52Z) - Whose Side Are You On? Investigating the Political Stance of Large Language Models [56.883423489203786]
We investigate the political orientation of Large Language Models (LLMs) across a spectrum of eight polarizing topics.
Our investigation delves into the political alignment of LLMs across a spectrum of eight polarizing topics, spanning from abortion to LGBTQ issues.
The findings suggest that users should be mindful when crafting queries, and exercise caution in selecting neutral prompt language.
arXiv Detail & Related papers (2024-03-15T04:02:24Z) - Behind the Screen: Investigating ChatGPT's Dark Personality Traits and
Conspiracy Beliefs [0.0]
This paper analyzes the dark personality traits and conspiracy beliefs of GPT-3.5 and GPT-4.
Dark personality traits and conspiracy beliefs were not particularly pronounced in either model.
arXiv Detail & Related papers (2024-02-06T16:03:57Z) - Primacy Effect of ChatGPT [69.49920102917598]
We study the primacy effect of ChatGPT: the tendency of selecting the labels at earlier positions as the answer.
We hope that our experiments and analyses provide additional insights into building more reliable ChatGPT-based solutions.
arXiv Detail & Related papers (2023-10-20T00:37:28Z) - AI in the Gray: Exploring Moderation Policies in Dialogic Large Language
Models vs. Human Answers in Controversial Topics [7.444967232238375]
Controversial topics, such as "religion", "gender identity", "freedom of speech", and "equality", can be a source of conflict.
By exposing ChatGPT to such debatable questions, we aim to understand its level of awareness and if existing models are subject to socio-political and/or economic biases.
arXiv Detail & Related papers (2023-08-28T14:23:04Z) - Large Language Models are not Fair Evaluators [60.27164804083752]
We find that the quality ranking of candidate responses can be easily hacked by altering their order of appearance in the context.
This manipulation allows us to skew the evaluation result, making one model appear considerably superior to the other.
We propose a framework with three simple yet effective strategies to mitigate this issue.
arXiv Detail & Related papers (2023-05-29T07:41:03Z) - Can ChatGPT Assess Human Personalities? A General Evaluation Framework [70.90142717649785]
Large Language Models (LLMs) have produced impressive results in various areas, but their potential human-like psychology is still largely unexplored.
This paper presents a generic evaluation framework for LLMs to assess human personalities based on Myers Briggs Type Indicator (MBTI) tests.
arXiv Detail & Related papers (2023-03-01T06:16:14Z) - BiasTestGPT: Using ChatGPT for Social Bias Testing of Language Models [73.29106813131818]
bias testing is currently cumbersome since the test sentences are generated from a limited set of manual templates or need expensive crowd-sourcing.
We propose using ChatGPT for the controllable generation of test sentences, given any arbitrary user-specified combination of social groups and attributes.
We present an open-source comprehensive bias testing framework (BiasTestGPT), hosted on HuggingFace, that can be plugged into any open-source PLM for bias testing.
arXiv Detail & Related papers (2023-02-14T22:07:57Z) - Diminished Diversity-of-Thought in a Standard Large Language Model [3.683202928838613]
We run replications of 14 studies from the Many Labs 2 replication project with OpenAI's text-davinci-003 model.
We find that among the eight studies we could analyse, our GPT sample replicated 37.5% of the original results and 37.5% of the Many Labs 2 results.
In one exploratory follow-up study, we found that a "correct answer" was robust to changing the demographic details that precede the prompt.
arXiv Detail & Related papers (2023-02-13T17:57:50Z) - The political ideology of conversational AI: Converging evidence on
ChatGPT's pro-environmental, left-libertarian orientation [0.0]
OpenAI introduced ChatGPT, a state-of-the-art dialogue model that can converse with its human counterparts.
This paper focuses on one of democratic society's most important decision-making processes: political elections.
We uncover ChatGPT's pro-environmental, left-libertarian ideology.
arXiv Detail & Related papers (2023-01-05T07:13:13Z) - The Tail Wagging the Dog: Dataset Construction Biases of Social Bias
Benchmarks [75.58692290694452]
We compare social biases with non-social biases stemming from choices made during dataset construction that might not even be discernible to the human eye.
We observe that these shallow modifications have a surprising effect on the resulting degree of bias across various models.
arXiv Detail & Related papers (2022-10-18T17:58:39Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.