Not All Pretraining are Created Equal: Threshold Tuning and Class Weighting for Imbalanced Polarization Tasks in Low-Resource Settings

• URL: http://arxiv.org/abs/2603.23534v1
• Date: Sun, 08 Mar 2026 15:50:37 GMT
• Title: Not All Pretraining are Created Equal: Threshold Tuning and Class Weighting for Imbalanced Polarization Tasks in Low-Resource Settings
• Abstract summary: This paper describes my submission to the Polarization Shared Task at SemEval-2025, which addresses polarization detection and classification in social media text.<n>I develop Transformer-based systems for English and Swahili across three subtasks: binary polarization detection, multi-label target type classification, and multi-label manifestation identification.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper describes my submission to the Polarization Shared Task at SemEval-2025, which addresses polarization detection and classification in social media text. I develop Transformer-based systems for English and Swahili across three subtasks: binary polarization detection, multi-label target type classification, and multi-label manifestation identification. The approach leverages multilingual and African language-specialized models (mDeBERTa-v3-base, SwahBERT, AfriBERTa-large), class-weighted loss functions, iterative stratified data splitting, and per-label threshold tuning to handle severe class imbalance. The best configuration, mDeBERTa-v3-base, achieves 0.8032 macro-F1 on validation for binary detection, with competitive performance on multi-label tasks (up to 0.556 macro-F1). Error analysis reveals persistent challenges with implicit polarization, code-switching, and distinguishing heated political discourse from genuine polarization.

Related papers

• Multilingual Polarization Detection Using Transformer-Based Models with Class Weighting and Threshold Tuning [0.0]
  We address binary polarization detection, polarization type classification, and manifestation identification for English and Swahili.<n>Our approach leverages transformer-based models with class-weighted loss functions to address severe label imbalance.<n>Our error analysis reveals that models struggle with dehumanization detection and lack of empathy.
  arXiv  Detail & Related papers  (2026-06-29T19:42:52Z)
• Lingo_Research_Group at SemEval-2026 Task 9: Evaluating Prompt Variants for Polarization Detection [1.921560550802477]
  We present a submission for SemEval-2026 Task 9: Multilingual Text Classification Challenge - Polarization Detection.<n>It covers all three subtasks: (1) binary polarization detection, (2) polarization type classification and (3) polarization manifestation identification.<n>We adopt a systematic approach of research on short designed prompts by considering twelve designed prompts that are different in terminology clarity.
  arXiv  Detail & Related papers  (2026-06-02T08:41:54Z)
• YEZE at SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization via Heterogeneous Ensembling [3.3438708071539445]
  We propose a heterogeneous ensemble of multilingual pretrained models, combining XLM-RoBERTa-large and mDeBERTa-v3-base.<n>We investigate techniques such as multi-task learning, translation-based data augmentation, and class weighting to improve classification performance under severe label imbalance.
  arXiv  Detail & Related papers  (2026-05-07T13:21:40Z)
• BITS Pilani at SemEval-2026 Task 9: Structured Supervised Fine-Tuning with DPO Refinement for Polarization Detection [2.2588605422113606]
  We present a two-stage approach for detecting political polarization in social media text.<n>We fine-tune Qwen 2.5-7B-Instruct with LoRA using an interpretable slot-filling template.<n>Experiments on the SemEval 2026 POLAR shared task dataset show that preference-based refinement improves both accuracy and decreases false negatives without extra annotation.
  arXiv  Detail & Related papers  (2026-04-13T07:35:17Z)
• SemEval-2026 Task 9: Detecting Multilingual, Multicultural and Multievent Online Polarization [47.011443052131376]
  We present SemEval-2026 Task 9, a shared task on online polarization detection, covering 22 languages and comprising over 110K annotated instances.<n>Participants were asked to predict labels in three sub-tasks: (1) detecting the presence of polarization, (2) identifying the type of polarization, and (3) recognizing the polarization manifestation.<n>We report the baseline results and analyze the performance of the best-performing systems, highlighting the most common approaches and the most effective methods across different subtasks and languages.
  arXiv  Detail & Related papers  (2026-04-08T08:34:55Z)
• SERSEM: Selective Entropy-Weighted Scoring for Membership Inference in Code Language Models [1.327416973220814]
  We propose SERSEM, a novel white-box attack framework that suppresses uninformative syntactical boilerplate to amplify memorization signals.<n>Our findings demonstrate that focusing on human-centric coding anomalies provides a significantly more robust indicator of verbatim memorization than sequence-level probability averages.
  arXiv  Detail & Related papers  (2026-04-01T17:03:58Z)
• MultiMatch: Multihead Consistency Regularization Matching for Semi-Supervised Text Classification [41.135013117834795]
  We introduce MultiMatch, a novel semi-supervised learning (SSL) algorithm combining the paradigms of co-training and consistency regularization with pseudo-labeling.<n>At its core, MultiMatch features a pseudo-label weighting module designed for selecting and filtering pseudo-labels based on head agreement and model confidence.
  arXiv  Detail & Related papers  (2025-06-09T14:27:47Z)
• Adaptive Thresholding for Multi-Label Classification via Global-Local Signal Fusion [0.0]
  Multi-label classification (MLC) requires predicting multiple labels per sample, often under heavy class imbalance and noisy conditions.<n>Traditional approaches apply fixed thresholds or treat labels independently, overlooking context and global rarity.<n>We introduce an adaptive thresholding mechanism that fuses global (IDF-based) and local (KNN-based) signals to produce per-label, per-instance thresholds.
  arXiv  Detail & Related papers  (2025-05-06T02:19:37Z)
• Project-Probe-Aggregate: Efficient Fine-Tuning for Group Robustness [61.45587642780908]
  We propose a three-step approach for parameter-efficient fine-tuning of image-text foundation models.<n>Our method improves its two key components: minority samples identification and the robust training algorithm.<n>Our theoretical analysis shows that our PPA enhances minority group identification and is Bayes optimal for minimizing the balanced group error.
  arXiv  Detail & Related papers  (2025-03-12T15:46:12Z)
• Responsible AI in NLP: GUS-Net Span-Level Bias Detection Dataset and Benchmark for Generalizations, Unfairness, and Stereotypes [6.30817290125825]
  We introduce the GUS-Net Framework, comprising the GUS dataset and a multi-label token-level detector for span-level analysis of social bias.<n>The GUS dataset contains 3,739 unique snippets across multiple domains, with over 69,000 token-level annotations.<n>We formulate bias detection as multi-label token-level classification and benchmark both encoder-based models and decoder-based large language models.
  arXiv  Detail & Related papers  (2024-10-10T21:51:22Z)
• Co-training for Low Resource Scientific Natural Language Inference [65.37685198688538]
  We propose a novel co-training method that assigns weights based on the training dynamics of the classifiers to the distantly supervised labels. By assigning importance weights instead of filtering out examples based on an arbitrary threshold on the predicted confidence, we maximize the usage of automatically labeled data. The proposed method obtains an improvement of 1.5% in Macro F1 over the distant supervision baseline, and substantial improvements over several other strong SSL baselines.
  arXiv  Detail & Related papers  (2024-06-20T18:35:47Z)
• Revisiting Domain-Adaptive 3D Object Detection by Reliable, Diverse and Class-balanced Pseudo-Labeling [38.07637524378327]
  Unsupervised domain adaptation (DA) with the aid of pseudo labeling techniques has emerged as a crucial approach for domain-adaptive 3D object detection. Existing DA methods suffer from a substantial drop in performance when applied to a multi-class training setting. We propose a novel ReDB framework tailored for learning to detect all classes at once.
  arXiv  Detail & Related papers  (2023-07-16T04:34:11Z)
• A Multi-level Supervised Contrastive Learning Framework for Low-Resource Natural Language Inference [54.678516076366506]
  Natural Language Inference (NLI) is a growingly essential task in natural language understanding. Here we propose a multi-level supervised contrastive learning framework named MultiSCL for low-resource natural language inference.
  arXiv  Detail & Related papers  (2022-05-31T05:54:18Z)
• PLM: Partial Label Masking for Imbalanced Multi-label Classification [59.68444804243782]
  Neural networks trained on real-world datasets with long-tailed label distributions are biased towards frequent classes and perform poorly on infrequent classes. We propose a method, Partial Label Masking (PLM), which utilizes this ratio during training. Our method achieves strong performance when compared to existing methods on both multi-label (MultiMNIST and MSCOCO) and single-label (imbalanced CIFAR-10 and CIFAR-100) image classification datasets.
  arXiv  Detail & Related papers  (2021-05-22T18:07:56Z)
• Rank-Consistency Deep Hashing for Scalable Multi-Label Image Search [90.30623718137244]
  We propose a novel deep hashing method for scalable multi-label image search. A new rank-consistency objective is applied to align the similarity orders from two spaces. A powerful loss function is designed to penalize the samples whose semantic similarity and hamming distance are mismatched.
  arXiv  Detail & Related papers  (2021-02-02T13:46:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.