Scaling Crowdsourced Election Monitoring: Construction and Evaluation of Classification Models for Multilingual and Cross-Domain Classification Settings

• URL: http://arxiv.org/abs/2503.03582v1
• Date: Wed, 05 Mar 2025 15:17:18 GMT
• Title: Scaling Crowdsourced Election Monitoring: Construction and Evaluation of Classification Models for Multilingual and Cross-Domain Classification Settings
• Authors: Jabez Magomere, Scott Hale,
• Abstract summary: We propose a two-step classification approach of first identifying informative reports and then categorising them into distinct information types.<n>We conduct classification experiments using multilingual transformer models such as XLM-RoBERTa and multilingual embeddings such as SBERT.<n>Our results show promising potential for model transfer across electoral domains, with F1-Scores of 59% in zero-shot and 63% in few-shot settings.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The adoption of crowdsourced election monitoring as a complementary alternative to traditional election monitoring is on the rise. Yet, its reliance on digital response volunteers to manually process incoming election reports poses a significant scaling bottleneck. In this paper, we address the challenge of scaling crowdsourced election monitoring by advancing the task of automated classification of crowdsourced election reports to multilingual and cross-domain classification settings. We propose a two-step classification approach of first identifying informative reports and then categorising them into distinct information types. We conduct classification experiments using multilingual transformer models such as XLM-RoBERTa and multilingual embeddings such as SBERT, augmented with linguistically motivated features. Our approach achieves F1-Scores of 77\% for informativeness detection and 75\% for information type classification. We conduct cross-domain experiments, applying models trained in a source electoral domain to a new target electoral domain in zero-shot and few-shot classification settings. Our results show promising potential for model transfer across electoral domains, with F1-Scores of 59\% in zero-shot and 63\% in few-shot settings. However, our analysis also reveals a performance bias in detecting informative English reports over Swahili, likely due to imbalances in the training data, indicating a need for caution when deploying classification models in real-world election scenarios.

Related papers

• Adaptation Method for Misinformation Identification [8.581136866856255]
  We propose ADOSE, an Active Domain Adaptation (ADA) framework for multimodal fake news detection. ADOSE actively annotates a small subset of target samples to improve detection performance. ADOSE outperforms existing ADA methods by 2.72% $sim$ 14.02%, indicating the superiority of our model.
  arXiv  Detail & Related papers  (2025-04-19T04:18:32Z)
• ElectionSim: Massive Population Election Simulation Powered by Large Language Model Driven Agents [70.17229548653852]
  We introduce ElectionSim, an innovative election simulation framework based on large language models. We present a million-level voter pool sampled from social media platforms to support accurate individual simulation. We also introduce PPE, a poll-based presidential election benchmark to assess the performance of our framework under the U.S. presidential election scenario.
  arXiv  Detail & Related papers  (2024-10-28T05:25:50Z)
• Automated stance detection in complex topics and small languages: the challenging case of immigration in polarizing news media [0.0]
  This paper explores the applicability of large language models for automated stance detection in a challenging scenario. It involves a morphologically complex, lower-resource language, and a socio-culturally complex topic, immigration. If the approach works in this case, it can be expected to perform as well or better in less demanding scenarios.
  arXiv  Detail & Related papers  (2023-05-22T13:56:35Z)
• Enhancing Pashto Text Classification using Language Processing Techniques for Single And Multi-Label Analysis [0.0]
  This study aims to establish an automated classification system for Pashto text. The study achieved an average testing accuracy rate of 94%. The use of pre-trained language representation models, such as DistilBERT, showed promising results.
  arXiv  Detail & Related papers  (2023-05-04T23:11:31Z)
• Explaining Cross-Domain Recognition with Interpretable Deep Classifier [100.63114424262234]
  Interpretable Deep (IDC) learns the nearest source samples of a target sample as evidence upon which the classifier makes the decision. Our IDC leads to a more explainable model with almost no accuracy degradation and effectively calibrates classification for optimum reject options.
  arXiv  Detail & Related papers  (2022-11-15T15:58:56Z)
• A Hierarchical Model for Spoken Language Recognition [29.948719321162883]
  Spoken language recognition ( SLR) refers to the automatic process used to determine the language present in a speech sample. We propose a novel hierarchical approach were two PLDA models are trained, one to generate scores for clusters of highly related languages and a second one to generate scores conditional to each cluster. We show that this hierarchical approach consistently outperforms the non-hierarchical one for detection of highly related languages.
  arXiv  Detail & Related papers  (2022-01-04T22:10:36Z)
• Genre as Weak Supervision for Cross-lingual Dependency Parsing [18.755176247223616]
  genre labels are frequently available, yet remain largely unexplored in cross-lingual setups. We project treebank-level genre information to the finer-grained sentence level. For 12 low-resource language treebanks, six of which are test-only, our genre-specific methods significantly outperform competitive baselines.
  arXiv  Detail & Related papers  (2021-09-10T08:24:54Z)
• Instance Level Affinity-Based Transfer for Unsupervised Domain Adaptation [74.71931918541748]
  We propose an instance affinity based criterion for source to target transfer during adaptation, called ILA-DA. We first propose a reliable and efficient method to extract similar and dissimilar samples across source and target, and utilize a multi-sample contrastive loss to drive the domain alignment process. We verify the effectiveness of ILA-DA by observing consistent improvements in accuracy over popular domain adaptation approaches on a variety of benchmark datasets.
  arXiv  Detail & Related papers  (2021-04-03T01:33:14Z)
• Unsupervised Domain Adaptation of a Pretrained Cross-Lingual Language Model [58.27176041092891]
  Recent research indicates that pretraining cross-lingual language models on large-scale unlabeled texts yields significant performance improvements. We propose a novel unsupervised feature decomposition method that can automatically extract domain-specific features from the entangled pretrained cross-lingual representations. Our proposed model leverages mutual information estimation to decompose the representations computed by a cross-lingual model into domain-invariant and domain-specific parts.
  arXiv  Detail & Related papers  (2020-11-23T16:00:42Z)
• LOGAN: Local Group Bias Detection by Clustering [86.38331353310114]
  We argue that evaluating bias at the corpus level is not enough for understanding how biases are embedded in a model. We propose LOGAN, a new bias detection technique based on clustering. Experiments on toxicity classification and object classification tasks show that LOGAN identifies bias in a local region.
  arXiv  Detail & Related papers  (2020-10-06T16:42:51Z)
• Joint Visual and Temporal Consistency for Unsupervised Domain Adaptive Person Re-Identification [64.37745443119942]
  This paper jointly enforces visual and temporal consistency in the combination of a local one-hot classification and a global multi-class classification. Experimental results on three large-scale ReID datasets demonstrate the superiority of proposed method in both unsupervised and unsupervised domain adaptive ReID tasks.
  arXiv  Detail & Related papers  (2020-07-21T14:31:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.