POLAR: A Benchmark for Multilingual, Multicultural, and Multi-Event Online Polarization

• URL: http://arxiv.org/abs/2505.20624v1
• Date: Tue, 27 May 2025 02:04:58 GMT
• Title: POLAR: A Benchmark for Multilingual, Multicultural, and Multi-Event Online Polarization
• Abstract summary: POLAR is a multilingual, multicultural, and multievent dataset with over 23k instances in seven languages.<n>We fine-tune six multilingual pretrained language models in both monolingual and cross-lingual setups.<n>We evaluate a range of open and closed large language models (LLMs) in few-shot and zero-shot scenarios.
• Score: 7.747535418644144
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Online polarization poses a growing challenge for democratic discourse, yet most computational social science research remains monolingual, culturally narrow, or event-specific. We introduce POLAR, a multilingual, multicultural, and multievent dataset with over 23k instances in seven languages from diverse online platforms and real-world events. Polarization is annotated along three axes: presence, type, and manifestation, using a variety of annotation platforms adapted to each cultural context. We conduct two main experiments: (1) we fine-tune six multilingual pretrained language models in both monolingual and cross-lingual setups; and (2) we evaluate a range of open and closed large language models (LLMs) in few-shot and zero-shot scenarios. Results show that while most models perform well on binary polarization detection, they achieve substantially lower scores when predicting polarization types and manifestations. These findings highlight the complex, highly contextual nature of polarization and the need for robust, adaptable approaches in NLP and computational social science. All resources will be released to support further research and effective mitigation of digital polarization globally.

Related papers

• Polarization Detection: A Hybrid Approach with AfroXLMR-Social and DeBERTa for Low- and High-Resource Settings [0.0]
  This paper presents our system description for the POLAR Task 2026, focusing on the detection and characterization of polarized discourse in English and Hausa.<n>We propose a hybrid modeling strategy for English binary detection, we leverage the monolingual strength of textbfDeBERTa.<n>We report competitive results across all three subtasks, demonstrating that model selection tailored to specific subtask requirements yields the best balance of performance.
  arXiv  Detail & Related papers  (2026-07-11T13:37:55Z)
• Lost in the Tower of Babel: The Adverse Effects of Incidental Multilingualism in LLMs [25.841183184960475]
  This paper argues that contemporary multilingual NLP has converged on a fragile and misleading paradigm of incidental multilingualism.<n>We contend that this paradigm systematically produces unequal, brittle, and opaque behavior across languages.<n>To address this, we call for a shift toward multilingualism by design.
  arXiv  Detail & Related papers  (2026-05-02T03:39:12Z)
• Bringing Up a Bilingual BabyLM: Investigating Multilingual Language Acquisition Using Small-Scale Models [2.475536483276974]
  We use language model training as a method for simulating a variety of highly controlled exposure conditions.<n>We create matched 100M-word mono- and bilingual datasets using synthetic data and machine translation.<n>Across model scales and measures, bilingual models perform similarly to monolingual models in one language, but show strong performance in the second language as well.
  arXiv  Detail & Related papers  (2026-03-31T10:32:25Z)
• Multilingual Text-to-Image Generation Magnifies Gender Stereotypes and Prompt Engineering May Not Help You [64.74707085021858]
  We show that multilingual models suffer from significant gender biases just as monolingual models do. We propose a novel benchmark, MAGBIG, intended to foster research on gender bias in multilingual models. Our results show that not only do models exhibit strong gender biases but they also behave differently across languages.
  arXiv  Detail & Related papers  (2024-01-29T12:02:28Z)
• Towards Bridging the Digital Language Divide [4.234367850767171]
  multilingual language processing systems often exhibit a hardwired, yet usually involuntary and hidden representational preference towards certain languages. We show that biased technology is often the result of research and development methodologies that do not do justice to the complexity of the languages being represented. We present a new initiative that aims at reducing linguistic bias through both technological design and methodology.
  arXiv  Detail & Related papers  (2023-07-25T10:53:20Z)
• Mitigating Data Imbalance and Representation Degeneration in Multilingual Machine Translation [103.90963418039473]
  Bi-ACL is a framework that uses only target-side monolingual data and a bilingual dictionary to improve the performance of the MNMT model. We show that Bi-ACL is more effective both in long-tail languages and in high-resource languages.
  arXiv  Detail & Related papers  (2023-05-22T07:31:08Z)
• TyDiP: A Dataset for Politeness Classification in Nine Typologically Diverse Languages [33.540256516320326]
  We study politeness phenomena in nine typologically diverse languages. We create TyDiP, a dataset containing three-way politeness annotations for 500 examples in each language.
  arXiv  Detail & Related papers  (2022-11-29T18:58:15Z)
• Universal and Independent: Multilingual Probing Framework for Exhaustive Model Interpretation and Evaluation [0.04199844472131922]
  We present and apply the GUI-assisted framework allowing us to easily probe a massive number of languages. Most of the regularities revealed in the mBERT model are typical for the western-European languages. Our framework can be integrated with the existing probing toolboxes, model cards, and leaderboards.
  arXiv  Detail & Related papers  (2022-10-24T13:41:17Z)
• Are Pretrained Multilingual Models Equally Fair Across Languages? [0.0]
  This work investigates the group fairness of multilingual models, asking whether these models are equally fair across languages. We evaluate three multilingual models on MozArt -- mBERT, XLM-R, and mT5 -- and show that across the four target languages, the three models exhibit different levels of group disparity.
  arXiv  Detail & Related papers  (2022-10-11T13:59:19Z)
• Towards Best Practices for Training Multilingual Dense Retrieval Models [54.91016739123398]
  We focus on the task of monolingual retrieval in a variety of typologically diverse languages using one such design. Our study is organized as a "best practices" guide for training multilingual dense retrieval models.
  arXiv  Detail & Related papers  (2022-04-05T17:12:53Z)
• Transformers in the loop: Polarity in neural models of language [0.20305676256390934]
  We probe polarity via so-called 'negative polarity items' in two pre-trained Transformer-based models. We show that metrics derived from language models are more consistent with data from psycholinguistic experiments than linguistic theory predictions.
  arXiv  Detail & Related papers  (2021-09-08T20:56:32Z)
• Discovering Representation Sprachbund For Multilingual Pre-Training [139.05668687865688]
  We generate language representation from multilingual pre-trained models and conduct linguistic analysis. We cluster all the target languages into multiple groups and name each group as a representation sprachbund. Experiments are conducted on cross-lingual benchmarks and significant improvements are achieved compared to strong baselines.
  arXiv  Detail & Related papers  (2021-09-01T09:32:06Z)
• Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer [101.58431011820755]
  We study gender bias in multilingual embeddings and how it affects transfer learning for NLP applications. We create a multilingual dataset for bias analysis and propose several ways for quantifying bias in multilingual representations.
  arXiv  Detail & Related papers  (2020-05-02T04:34:37Z)
• XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning [68.57658225995966]
  Cross-lingual Choice of Plausible Alternatives (XCOPA) is a typologically diverse multilingual dataset for causal commonsense reasoning in 11 languages. We evaluate a range of state-of-the-art models on this novel dataset, revealing that the performance of current methods falls short compared to translation-based transfer.
  arXiv  Detail & Related papers  (2020-05-01T12:22:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.