Fine-Grained Detection of Solidarity for Women and Migrants in 155 Years of German Parliamentary Debates

• URL: http://arxiv.org/abs/2210.04359v2
• Date: Mon, 24 Jun 2024 20:01:19 GMT
• Title: Fine-Grained Detection of Solidarity for Women and Migrants in 155 Years of German Parliamentary Debates
• Authors: Aida Kostikova, Benjamin Paassen, Dominik Beese, Ole Pütz, Gregor Wiedemann, Steffen Eger,
• Abstract summary: We study solidarity towards women and migrants in German parliamentary debates between 1867 and 2022. Using 2,864 manually annotated text snippets, we evaluate large language models (LLMs) like Llama 3, GPT-3.5, and GPT-4. Using GPT-4, we automatically annotate more than 18k further instances (with a cost of around 500 Euro) across 155 years.
• Score: 16.382860418871804
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Solidarity is a crucial concept to understand social relations in societies. In this paper, we explore fine-grained solidarity frames to study solidarity towards women and migrants in German parliamentary debates between 1867 and 2022. Using 2,864 manually annotated text snippets (with a cost exceeding 18k Euro), we evaluate large language models (LLMs) like Llama 3, GPT-3.5, and GPT-4. We find that GPT-4 outperforms other LLMs, approaching human annotation quality. Using GPT-4, we automatically annotate more than 18k further instances (with a cost of around 500 Euro) across 155 years and find that solidarity with migrants outweighs anti-solidarity but that frequencies and solidarity types shift over time. Most importantly, group-based notions of (anti-)solidarity fade in favor of compassionate solidarity, focusing on the vulnerability of migrant groups, and exchange-based anti-solidarity, focusing on the lack of (economic) contribution. Our study highlights the interplay of historical events, socio-economic needs, and political ideologies in shaping migration discourse and social cohesion. We also show that powerful LLMs, if carefully prompted, can be cost-effective alternatives to human annotation for hard social scientific tasks.

Related papers

• Large Language Models Reflect the Ideology of their Creators [73.25935570218375]
  Large language models (LLMs) are trained on vast amounts of data to generate natural language. We uncover notable diversity in the ideological stance exhibited across different LLMs and languages.
  arXiv  Detail & Related papers  (2024-10-24T04:02:30Z)
• Arabic Dataset for LLM Safeguard Evaluation [62.96160492994489]
  This study explores the safety of large language models (LLMs) in Arabic with its linguistic and cultural complexities. We present an Arab-region-specific safety evaluation dataset consisting of 5,799 questions, including direct attacks, indirect attacks, and harmless requests with sensitive words.
  arXiv  Detail & Related papers  (2024-10-22T14:12:43Z)
• Are Social Sentiments Inherent in LLMs? An Empirical Study on Extraction of Inter-demographic Sentiments [14.143299702954023]
  This study focuses on social groups defined in terms of nationality, religion, and race/ethnicity. We input questions regarding sentiments from one group to another into LLMs, apply sentiment analysis to the responses, and compare the results with social surveys.
  arXiv  Detail & Related papers  (2024-08-08T08:13:25Z)
• In Silico Sociology: Forecasting COVID-19 Polarization with Large Language Models [0.0]
  We train deep neural networks on massive archives of digitized text to learn the complex linguistic patterns that constitute historic and contemporary discourses. We reconstruct the public opinion landscape of 2019 to examine the extent to which the future polarization over COVID-19 was prefigured in existing political discourse. We find that the simulated respondents reproduce observed partisan differences in COVID-19 attitudes in 84% of cases, significantly greater than chance.
  arXiv  Detail & Related papers  (2024-05-23T22:10:12Z)
• The Call for Socially Aware Language Technologies [94.6762219597438]
  We argue that many of these issues share a common core: a lack of awareness of the factors, context, and implications of the social environment in which NLP operates. We argue that substantial challenges remain for NLP to develop social awareness and that we are just at the beginning of a new era for the field.
  arXiv  Detail & Related papers  (2024-05-03T18:12:39Z)
• AraTrust: An Evaluation of Trustworthiness for LLMs in Arabic [0.0]
  We introduce AraTrust, the first comprehensive trustworthiness benchmark for Large Language Models (LLMs) in Arabic. GPT-4 was the most trustworthy LLM, while open-source models, particularly AceGPT 7B and Jais 13B, struggled to achieve a score of 60% in our benchmark.
  arXiv  Detail & Related papers  (2024-03-14T00:45:24Z)
• Large Language Models are Not Yet Human-Level Evaluators for Abstractive Summarization [66.08074487429477]
  We investigate the stability and reliability of large language models (LLMs) as automatic evaluators for abstractive summarization. We find that while ChatGPT and GPT-4 outperform the commonly used automatic metrics, they are not ready as human replacements.
  arXiv  Detail & Related papers  (2023-05-22T14:58:13Z)
• Consistency Analysis of ChatGPT [65.268245109828]
  This paper investigates the trustworthiness of ChatGPT and GPT-4 regarding logically consistent behaviour. Our findings suggest that while both models appear to show an enhanced language understanding and reasoning ability, they still frequently fall short of generating logically consistent predictions.
  arXiv  Detail & Related papers  (2023-03-11T01:19:01Z)
• Changes in European Solidarity Before and During COVID-19: Evidence from a Large Crowd- and Expert-Annotated Twitter Dataset [77.27709662210363]
  We introduce the well-established social scientific concept of social solidarity and its contestation, anti-solidarity, as a new problem setting to supervised machine learning in NLP. We annotate 2.3k English and German tweets for (anti-)solidarity expressions, utilizing multiple human annotators and two annotation approaches (experts vs. crowds) Our results show that solidarity became increasingly salient and contested during the COVID-19 crisis.
  arXiv  Detail & Related papers  (2021-08-02T17:03:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.