Two-Stage Bengali Sentiment Classification: Domain Adaptation Through Continual Learning and Parameter-Efficient Fine-Tuning

• URL: http://arxiv.org/abs/2608.01471v1
• Date: Sun, 02 Aug 2026 20:07:56 GMT
• Title: Two-Stage Bengali Sentiment Classification: Domain Adaptation Through Continual Learning and Parameter-Efficient Fine-Tuning
• Abstract summary: SentiBanglaBERT is a two-stage Bengali sentiment classification framework.<n>It combines domain-adaptive continual pretraining and parameter-efficient fine-tuning.<n>It offers linguistic insights into how Bengali morphological cues, such as negation suffixes and aspectual markers, influence sentiment predictions.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Understanding sentiment in low-resource languages remains a key challenge for Natural Language Processing (NLP), particularly when domain-specific data is scarce. In this work, we present SentiBanglaBERT, a two-stage Bengali sentiment classification framework combining domain-adaptive continual pretraining and parameter-efficient fine-tuning. The approach enables contextual adaptation to news-style data while remaining computationally efficient through Low-Rank Adaptation (LoRA). Beyond performance, SentiBanglaBERT integrates SHAP-based interpretability, offering linguistic insights into how Bengali morphological cues, such as negation suffixes and aspectual markers, influence sentiment predictions. Experiments demonstrate stable performance comparable to strong baselines while providing greater transparency and interpretive depth. This framework highlights the potential of domain-adaptive continual learning as a foundation for interpretable, resource-efficient NLP in morphologically rich, underrepresented languages.

Related papers

• Translation as a Computationally Efficient Bridge: Feasibility of English BERT for Low-Resource Languages [1.3899485276817554]
  BERT models have revolutionised Natural Language Processing (NLP) through their ability to process unstructured text across diverse domains.<n>High-quality BERT models for non-English languages remains challenging due to limited annotated data and high computational demands.<n>This study compares translation-based fine-tuning with native-language BERT performance across tasks and languages.
  arXiv  Detail & Related papers  (2026-07-14T10:44:45Z)
• LaScA: Language-Conditioned Scalable Modelling of Affective Dynamics [3.3187704612685267]
  We propose a novel framework that uses Language Models (LMs) as semantic context conditioners over handcrafted affect descriptors.<n>Our approach begins with interpretable facial geometry and acoustic features derived from structured domain knowledge.<n>A pretrained LM processes these descriptions to generate semantic context embeddings that act as high-level priors over affective dynamics.
  arXiv  Detail & Related papers  (2026-04-08T15:18:39Z)
• Compensating Visual Insufficiency with Stratified Language Guidance for Long-Tail Class Incremental Learning [50.795452298625996]
  Long-tail class incremental learning (LT CIL) remains highly challenging because the scarcity of samples in tail classes hampers their learning.<n>We exploit the informativeness and scalability of language knowledge to tackle these issues.<n>We introduce stratified adaptive language guidance, which leverages learnable weights to merge multi-scale semantic representations.
  arXiv  Detail & Related papers  (2026-03-23T08:50:00Z)
• Tokenization and Morphological Fidelity in Uralic NLP: A Cross-Lingual Evaluation [9.23725598061561]
  This study systematically compares three subword paradigms -- Byte Pair.<n>(BPE), Overlap BPE (OBPE), and Unigram Language Model -- across six Uralic languages.<n>We show OBPE consistently achieves stronger morphological alignment and higher tagging accuracy than conventional methods.
  arXiv  Detail & Related papers  (2026-02-04T05:59:25Z)
• Unraveling the Dominance of Large Language Models Over Transformer Models for Bangla Natural Language Inference: A Comprehensive Study [0.0]
  Natural Language Inference (NLI) is a cornerstone of Natural Language Processing (NLP) This study addresses the underexplored area of evaluating Large Language Models (LLMs) in low-resourced languages like Bengali.
  arXiv  Detail & Related papers  (2024-05-05T13:57:05Z)
• Natural Language Processing for Dialects of a Language: A Survey [56.93337350526933]
  State-of-the-art natural language processing (NLP) models are trained on massive training corpora, and report a superlative performance on evaluation datasets.<n>This survey delves into an important attribute of these datasets: the dialect of a language.<n>Motivated by the performance degradation of NLP models for dialectal datasets and its implications for the equity of language technologies, we survey past research in NLP for dialects in terms of datasets, and approaches.
  arXiv  Detail & Related papers  (2024-01-11T03:04:38Z)
• Improving Language Models Meaning Understanding and Consistency by Learning Conceptual Roles from Dictionary [65.268245109828]
  Non-human-like behaviour of contemporary pre-trained language models (PLMs) is a leading cause undermining their trustworthiness. A striking phenomenon is the generation of inconsistent predictions, which produces contradictory results. We propose a practical approach that alleviates the inconsistent behaviour issue by improving PLM awareness.
  arXiv  Detail & Related papers  (2023-10-24T06:15:15Z)
• A Multi-level Supervised Contrastive Learning Framework for Low-Resource Natural Language Inference [54.678516076366506]
  Natural Language Inference (NLI) is a growingly essential task in natural language understanding. Here we propose a multi-level supervised contrastive learning framework named MultiSCL for low-resource natural language inference.
  arXiv  Detail & Related papers  (2022-05-31T05:54:18Z)
• To Augment or Not to Augment? A Comparative Study on Text Augmentation Techniques for Low-Resource NLP [0.0]
  We investigate three categories of text augmentation methodologies which perform changes on the syntax. We compare them on part-of-speech tagging, dependency parsing and semantic role labeling for a diverse set of language families. Our results suggest that the augmentation techniques can further improve over strong baselines based on mBERT.
  arXiv  Detail & Related papers  (2021-11-18T10:52:48Z)
• ERICA: Improving Entity and Relation Understanding for Pre-trained Language Models via Contrastive Learning [97.10875695679499]
  We propose a novel contrastive learning framework named ERICA in pre-training phase to obtain a deeper understanding of the entities and their relations in text. Experimental results demonstrate that our proposed ERICA framework achieves consistent improvements on several document-level language understanding tasks.
  arXiv  Detail & Related papers  (2020-12-30T03:35:22Z)
• TAVAT: Token-Aware Virtual Adversarial Training for Language Understanding [55.16953347580948]
  Gradient-based adversarial training is widely used in improving the robustness of neural networks. It cannot be easily adapted to natural language processing tasks since the embedding space is discrete. We propose a Token-Aware Virtual Adrial Training method to craft fine-grained perturbations.
  arXiv  Detail & Related papers  (2020-04-30T02:03:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.