Fake News Classification in Urdu: A Domain Adaptation Approach for a Low-Resource Language

• URL: http://arxiv.org/abs/2512.22778v1
• Date: Sun, 28 Dec 2025 04:42:40 GMT
• Title: Fake News Classification in Urdu: A Domain Adaptation Approach for a Low-Resource Language
• Authors: Muhammad Zain Ali, Bernhard Pfahringer, Tony Smith,
• Abstract summary: We investigate the effectiveness of domain adaptation before fine-tuning for fake news classification in Urdu.<n>We evaluate two widely used multilingual models, XLM-RoBERTa and mBERT, and apply domain-adaptive pretraining.<n>Experiments on four publicly available Urdu fake news datasets show that domain-adapted XLM-R consistently outperforms its vanilla counterpart.
• Score: 4.982413436059687
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Misinformation on social media is a widely acknowledged issue, and researchers worldwide are actively engaged in its detection. However, low-resource languages such as Urdu have received limited attention in this domain. An obvious approach is to utilize a multilingual pretrained language model and fine-tune it for a downstream classification task, such as misinformation detection. However, these models struggle with domain-specific terms, leading to suboptimal performance. To address this, we investigate the effectiveness of domain adaptation before fine-tuning for fake news classification in Urdu, employing a staged training approach to optimize model generalization. We evaluate two widely used multilingual models, XLM-RoBERTa and mBERT, and apply domain-adaptive pretraining using a publicly available Urdu news corpus. Experiments on four publicly available Urdu fake news datasets show that domain-adapted XLM-R consistently outperforms its vanilla counterpart, while domain-adapted mBERT exhibits mixed results.

Related papers

• Unified Large Language Models for Misinformation Detection in Low-Resource Linguistic Settings [1.5811829698567754]
  There is a noticeable gap in resources and strategies to detect news in regional languages, such as Urdu.<n>Current Urdu fake news datasets are often domain-specific and inaccessible to the public.<n>This paper presents the first benchmark large FND dataset for Urdu news, which is publicly available for validation and deep analysis.
  arXiv  Detail & Related papers  (2025-06-02T12:19:28Z)
• Multi-stage Training of Bilingual Islamic LLM for Neural Passage Retrieval [0.0]
  The research employs a language reduction technique to create a lightweight bilingual large language model (LLM)<n>Our approach for domain adaptation addresses the unique challenges faced in the Islamic domain, where substantial in-domain corpora exist only in Arabic.<n>The findings suggest that combining domain adaptation and a multi-stage training method for the bilingual Islamic neural retrieval model enables it to outperform monolingual models on downstream retrieval tasks.
  arXiv  Detail & Related papers  (2025-01-17T13:17:42Z)
• Paloma: A Benchmark for Evaluating Language Model Fit [112.481957296585]
  Evaluations of language models (LMs) commonly report perplexity on monolithic data held out from training.<n>We introduce Perplexity Analysis for Language Model Assessment (Paloma), a benchmark to measure LM fit to 546 English and code domains.
  arXiv  Detail & Related papers  (2023-12-16T19:12:45Z)
• Domain Private Transformers for Multi-Domain Dialog Systems [2.7013801448234367]
  This paper proposes domain privacy as a novel way to quantify how likely a conditional language model will leak across domains. Experiments on membership inference attacks show that our proposed method has comparable resiliency to methods adapted from recent literature on differentially private language models.
  arXiv  Detail & Related papers  (2023-05-23T16:27:12Z)
• SwitchPrompt: Learning Domain-Specific Gated Soft Prompts for Classification in Low-Resource Domains [14.096170976149521]
  SwitchPrompt is a novel and lightweight prompting methodology for adaptation of language models trained on datasets from the general domain to diverse low-resource domains. Our few-shot experiments on three text classification benchmarks demonstrate the efficacy of the general-domain pre-trained language models when used with SwitchPrompt. They often even outperform their domain-specific counterparts trained with baseline state-of-the-art prompting methods by up to 10.7% performance increase in accuracy.
  arXiv  Detail & Related papers  (2023-02-14T07:14:08Z)
• Improving Pre-trained Language Model Fine-tuning with Noise Stability Regularization [94.4409074435894]
  We propose a novel and effective fine-tuning framework, named Layerwise Noise Stability Regularization (LNSR) Specifically, we propose to inject the standard Gaussian noise and regularize hidden representations of the fine-tuned model. We demonstrate the advantages of the proposed method over other state-of-the-art algorithms including L2-SP, Mixout and SMART.
  arXiv  Detail & Related papers  (2022-06-12T04:42:49Z)
• Transformer-based Language Model Fine-tuning Methods for COVID-19 Fake News Detection [7.29381091750894]
  We propose a novel transformer-based language model fine-tuning approach for these fake news detection. First, the token vocabulary of individual model is expanded for the actual semantics of professional phrases. Last, the predicted features extracted by universal language model RoBERTa and domain-specific model CT-BERT are fused by one multiple layer perception to integrate fine-grained and high-level specific representations.
  arXiv  Detail & Related papers  (2021-01-14T09:05:42Z)
• UNKs Everywhere: Adapting Multilingual Language Models to New Scripts [103.79021395138423]
  Massively multilingual language models such as multilingual BERT (mBERT) and XLM-R offer state-of-the-art cross-lingual transfer performance on a range of NLP tasks. Due to their limited capacity and large differences in pretraining data, there is a profound performance gap between resource-rich and resource-poor target languages. We propose novel data-efficient methods that enable quick and effective adaptation of pretrained multilingual models to such low-resource languages and unseen scripts.
  arXiv  Detail & Related papers  (2020-12-31T11:37:28Z)
• Unsupervised Domain Adaptation of a Pretrained Cross-Lingual Language Model [58.27176041092891]
  Recent research indicates that pretraining cross-lingual language models on large-scale unlabeled texts yields significant performance improvements. We propose a novel unsupervised feature decomposition method that can automatically extract domain-specific features from the entangled pretrained cross-lingual representations. Our proposed model leverages mutual information estimation to decompose the representations computed by a cross-lingual model into domain-invariant and domain-specific parts.
  arXiv  Detail & Related papers  (2020-11-23T16:00:42Z)
• Comparison of Interactive Knowledge Base Spelling Correction Models for Low-Resource Languages [81.90356787324481]
  Spelling normalization for low resource languages is a challenging task because the patterns are hard to predict. This work shows a comparison of a neural model and character language models with varying amounts on target language data. Our usage scenario is interactive correction with nearly zero amounts of training examples, improving models as more data is collected.
  arXiv  Detail & Related papers  (2020-10-20T17:31:07Z)
• Dynamic Data Selection and Weighting for Iterative Back-Translation [116.14378571769045]
  We propose a curriculum learning strategy for iterative back-translation models. We evaluate our models on domain adaptation, low-resource, and high-resource MT settings. Experimental results demonstrate that our methods achieve improvements of up to 1.8 BLEU points over competitive baselines.
  arXiv  Detail & Related papers  (2020-04-07T19:49:58Z)
• Unsupervised Domain Clusters in Pretrained Language Models [61.832234606157286]
  We show that massive pre-trained language models implicitly learn sentence representations that cluster by domains without supervision. We propose domain data selection methods based on such models. We evaluate our data selection methods for neural machine translation across five diverse domains.
  arXiv  Detail & Related papers  (2020-04-05T06:22:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.