Assessing the Political Fairness of Multilingual LLMs: A Case Study based on a 21-way Multiparallel EuroParl Dataset

• URL: http://arxiv.org/abs/2510.20508v1
• Date: Thu, 23 Oct 2025 12:50:30 GMT
• Title: Assessing the Political Fairness of Multilingual LLMs: A Case Study based on a 21-way Multiparallel EuroParl Dataset
• Authors: Paul Lerner, François Yvon,
• Abstract summary: This study is made possible by a new, 21-way multiparallel version of EuroParl, the parliamentary proceedings of the EP.<n>The dataset consists of 1.5M sentences for a total of 40M words and 249M characters.<n>It covers three years, 1000+ speakers, 7 countries, 12 EU parties, 25 EU committees, and hundreds of national parties.
• Score: 19.75260643083416
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The political biases of Large Language Models (LLMs) are usually assessed by simulating their answers to English surveys. In this work, we propose an alternative framing of political biases, relying on principles of fairness in multilingual translation. We systematically compare the translation quality of speeches in the European Parliament (EP), observing systematic differences with majority parties from left, center, and right being better translated than outsider parties. This study is made possible by a new, 21-way multiparallel version of EuroParl, the parliamentary proceedings of the EP, which includes the political affiliations of each speaker. The dataset consists of 1.5M sentences for a total of 40M words and 249M characters. It covers three years, 1000+ speakers, 7 countries, 12 EU parties, 25 EU committees, and hundreds of national parties.

Related papers

• PART: Progressive Alignment Representation Training for Multilingual Speech-To-Text with LLMs [58.2469845374385]
  We introduce Progressive Alignment Representation Training (PART)<n>PART is a multi-stage and multi-task framework that separates within-language from cross-language alignment.<n>Experiments on CommonVoice 15, Fleurs, Wenetspeech, and CoVoST2 show that PART surpasses conventional approaches.
  arXiv  Detail & Related papers  (2025-09-24T03:54:14Z)
• Benchmarking Gender and Political Bias in Large Language Models [37.192287982246526]
  We introduce EuroParlVote, a novel benchmark for evaluating large language models (LLMs) in politically sensitive contexts.<n>It links European Parliament debate speeches to roll-call vote outcomes and includes rich demographic metadata for each Member of the European Parliament (MEP)<n>Using EuroParlVote, we evaluate state-of-the-art LLMs on two tasks -- gender classification and vote prediction -- revealing consistent patterns of bias.
  arXiv  Detail & Related papers  (2025-09-07T18:23:30Z)
• Large Language Models Reflect the Ideology of their Creators [71.65505524599888]
  Large language models (LLMs) are trained on vast amounts of data to generate natural language.<n>This paper shows that the ideological stance of an LLM appears to reflect the worldview of its creators.
  arXiv  Detail & Related papers  (2024-10-24T04:02:30Z)
• SpeakGer: A meta-data enriched speech corpus of German state and federal parliaments [0.12277343096128711]
  We provide the SpeakGer data set, consisting of German parliament debates from all 16 federal states of Germany as well as the German Bundestag from 1947-2023. This data set includes rich meta data in form of information on both reactions from the audience towards the speech as well as information about the speaker's party, their age, their constituency and their party's political alignment.
  arXiv  Detail & Related papers  (2024-10-23T14:00:48Z)
• Representation Bias in Political Sample Simulations with Large Language Models [54.48283690603358]
  This study seeks to identify and quantify biases in simulating political samples with Large Language Models. Using the GPT-3.5-Turbo model, we leverage data from the American National Election Studies, German Longitudinal Election Study, Zuobiao dataset, and China Family Panel Studies.
  arXiv  Detail & Related papers  (2024-07-16T05:52:26Z)
• Language Model Alignment in Multilingual Trolley Problems [138.5684081822807]
  Building on the Moral Machine experiment, we develop a cross-lingual corpus of moral dilemma vignettes in over 100 languages called MultiTP.<n>Our analysis explores the alignment of 19 different LLMs with human judgments, capturing preferences across six moral dimensions.<n>We discover significant variance in alignment across languages, challenging the assumption of uniform moral reasoning in AI systems.
  arXiv  Detail & Related papers  (2024-07-02T14:02:53Z)
• Llama meets EU: Investigating the European Political Spectrum through the Lens of LLMs [18.836470390824633]
  We audit Llama Chat in the context of EU politics to analyze the model's political knowledge and its ability to reason in context. We adapt, i.e., further fine-tune, Llama Chat on speeches of individual euro-parties from debates in the European Parliament to reevaluate its political leaning.
  arXiv  Detail & Related papers  (2024-03-20T13:42:57Z)
• Modelling Political Coalition Negotiations Using LLM-based Agents [53.934372246390495]
  We introduce coalition negotiations as a novel NLP task, and model it as a negotiation between large language model-based agents. We introduce a multilingual dataset, POLCA, comprising manifestos of European political parties and coalition agreements over a number of elections in these countries. We propose a hierarchical Markov decision process designed to simulate the process of coalition negotiation between political parties and predict the outcomes.
  arXiv  Detail & Related papers  (2024-02-18T21:28:06Z)
• Multi-EuP: The Multilingual European Parliament Dataset for Analysis of Bias in Information Retrieval [62.82448161570428]
  This dataset is designed to investigate fairness in a multilingual information retrieval context. It boasts an authentic multilingual corpus, featuring topics translated into all 24 languages. It offers rich demographic information associated with its documents, facilitating the study of demographic bias.
  arXiv  Detail & Related papers  (2023-11-03T12:29:11Z)
• The ParlaSent Multilingual Training Dataset for Sentiment Identification in Parliamentary Proceedings [0.0]
  The paper presents a new training dataset of sentences in 7 languages, manually annotated for sentiment. The paper additionally introduces the first domain-specific multilingual transformer language model for political science applications.
  arXiv  Detail & Related papers  (2023-09-18T14:01:06Z)
• Multi-aspect Multilingual and Cross-lingual Parliamentary Speech Analysis [1.759288298635146]
  We apply advanced NLP methods to a joint and comparative analysis of six national parliaments between 2017 and 2020. We analyze emotions and sentiment in the transcripts from the ParlaMint dataset collection. The results show some commonalities and many surprising differences among the analyzed countries.
  arXiv  Detail & Related papers  (2022-07-03T14:31:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.