Performance Evaluation of Emotion Classification in Japanese Using RoBERTa and DeBERTa

• URL: http://arxiv.org/abs/2505.00013v1
• Date: Tue, 22 Apr 2025 07:51:37 GMT
• Title: Performance Evaluation of Emotion Classification in Japanese Using RoBERTa and DeBERTa
• Abstract summary: Social media monitoring and customer-feedback analysis require accurate emotion detection for Japanese text.<n>This study aims to build a high-accuracy model for predicting the presence or absence of eight Plutchik emotions in Japanese sentences.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Background Practical applications such as social media monitoring and customer-feedback analysis require accurate emotion detection for Japanese text, yet resource scarcity and class imbalance hinder model performance. Objective This study aims to build a high-accuracy model for predicting the presence or absence of eight Plutchik emotions in Japanese sentences. Methods Using the WRIME corpus, we transform reader-averaged intensity scores into binary labels and fine-tune four pre-trained language models (BERT, RoBERTa, DeBERTa-v3-base, DeBERTa-v3-large). For context, we also assess two large language models (TinySwallow-1.5B-Instruct and ChatGPT-4o). Accuracy and F1-score serve as evaluation metrics. Results DeBERTa-v3-large attains the best mean accuracy (0.860) and F1-score (0.662), outperforming all other models. It maintains robust F1 across both high-frequency emotions (e.g., Joy, Anticipation) and low-frequency emotions (e.g., Anger, Trust). The LLMs lag, with ChatGPT-4o and TinySwallow-1.5B-Instruct scoring 0.527 and 0.292 in mean F1, respectively. Conclusion The fine-tuned DeBERTa-v3-large model currently offers the most reliable solution for binary emotion classification in Japanese. We release this model as a pip-installable package (pip install deberta-emotion-predictor). Future work should augment data for rare emotions, reduce model size, and explore prompt engineering to improve LLM performance. This manuscript is under review for possible publication in New Generation Computing.

Related papers

• Pushing on Multilingual Reasoning Models with Language-Mixed Chain-of-Thought [23.847410628315544]
  We introduce **Language-Mixed CoT**, a reasoning schema that switches between English and a target language.<n>We train ninve models (4B-35B) across six families (Qwen2.5, Llama-3.1, Gemma-3, etc)<n>Our best model, **KO-REAson-35B**, achieves state-of-the-art performance, with the highest overall average score (64.0 pm 25)
  arXiv  Detail & Related papers  (2025-10-05T14:39:41Z)
• A Comparative Evaluation of Large Language Models for Persian Sentiment Analysis and Emotion Detection in Social Media Texts [2.820011731460364]
  This study presents a comparative evaluation of four large language models (LLMs) for sentiment analysis and emotion detection in Persian social media texts.<n>The results show that all models reach an acceptable level of performance, and a statistical comparison of the best three models indicates no significant differences among them.<n>The findings indicate that the emotion detection task is more challenging for all models compared to the sentiment analysis task, and the misclassification patterns can represent some challenges in Persian language texts.
  arXiv  Detail & Related papers  (2025-09-18T12:59:07Z)
• XplaiNLP at CheckThat! 2025: Multilingual Subjectivity Detection with Finetuned Transformers and Prompt-Based Inference with Large Language Models [2.749729059235755]
  This notebook reports the Xplai submission to the CheckThat! 2025 shared task on multilingual subjectivity detection.<n>We evaluate two approaches: supervised fine-tuning of transformer encoders, EuroBERT, XLM-RoBERTa, and German-BERT, on monolingual and machine-translated training data.<n>For German, a German-BERT model fine-tuned on translated training data from typologically related languages yields competitive performance over the baseline.
  arXiv  Detail & Related papers  (2025-09-15T16:53:41Z)
• mmBERT: A Modern Multilingual Encoder with Annealed Language Learning [57.58071656545661]
  mmBERT is an encoder-only language model pretrained on 3T tokens of multilingual text.<n>We add over 1700 low-resource languages to the data mix only during the decay phase.<n>We show that mmBERT significantly outperforms the previous generation of models on classification and retrieval tasks.
  arXiv  Detail & Related papers  (2025-09-08T17:08:42Z)
• RideKE: Leveraging Low-Resource, User-Generated Twitter Content for Sentiment and Emotion Detection in Kenyan Code-Switched Dataset [4.142287865325998]
  We analyze Kenyan code-switched data and evaluate four state-of-the-art (SOTA) transformer-based pretrained models for sentiment and emotion classification.<n>For sentiment analysis, XLM-R supervised model achieves the highest accuracy (69.2%) and F1 score (66.1%), XLM-R semi-supervised (67.2% accuracy, 64.1% F1 score)<n>In emotion analysis, DistilBERT supervised leads in accuracy (59.8%) and F1 score (31%), mBERT semi-supervised (accuracy (59% and F1 score 26.5
  arXiv  Detail & Related papers  (2025-02-10T06:18:07Z)
• Xmodel-1.5: An 1B-scale Multilingual LLM [4.298869484709548]
  We introduce Xmodel-1.5, a multilingual large language model pretrained on 2 trillion tokens.<n>Xmodel-1.5 employs a custom unigram tokenizer with 65,280 tokens, optimizing both efficiency and accuracy.<n>The model delivers competitive results across multiple languages, including Thai, Arabic, French, Chinese, and English.
  arXiv  Detail & Related papers  (2024-11-15T10:01:52Z)
• Predictor-Corrector Enhanced Transformers with Exponential Moving Average Coefficient Learning [73.73967342609603]
  We introduce a predictor-corrector learning framework to minimize truncation errors. We also propose an exponential moving average-based coefficient learning method to strengthen our higher-order predictor. Our model surpasses a robust 3.8B DeepNet by an average of 2.9 SacreBLEU, using only 1/3 parameters.
  arXiv  Detail & Related papers  (2024-11-05T12:26:25Z)
• Training Language Models with Language Feedback at Scale [50.70091340506957]
  We introduce learning from Language Feedback (ILF), a new approach that utilizes more informative language feedback. ILF consists of three steps that are applied iteratively: first, conditioning the language model on the input, an initial LM output, and feedback to generate refinements. We show theoretically that ILF can be viewed as Bayesian Inference, similar to Reinforcement Learning from human feedback.
  arXiv  Detail & Related papers  (2023-03-28T17:04:15Z)
• Few-shot Learning with Multilingual Language Models [66.49496434282564]
  We train multilingual autoregressive language models on a balanced corpus covering a diverse set of languages. Our largest model sets new state of the art in few-shot learning in more than 20 representative languages. We present a detailed analysis of where the model succeeds and fails, showing in particular that it enables cross-lingual in-context learning.
  arXiv  Detail & Related papers  (2021-12-20T16:52:35Z)
• DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharing [117.41016786835452]
  This paper presents a new pre-trained language model, DeBERTaV3, which improves the original DeBERTa model. vanilla embedding sharing in ELECTRA hurts training efficiency and model performance. We propose a new gradient-disentangled embedding sharing method that avoids the tug-of-war dynamics.
  arXiv  Detail & Related papers  (2021-11-18T06:48:00Z)
• Towards Efficient NLP: A Standard Evaluation and A Strong Baseline [55.29756535335831]
  This work presents ELUE (Efficient Language Understanding Evaluation), a standard evaluation, and a public leaderboard for efficient NLP models. Along with the benchmark, we also pre-train and release a strong baseline, ElasticBERT, whose elasticity is both static and dynamic.
  arXiv  Detail & Related papers  (2021-10-13T21:17:15Z)
• HeBERT & HebEMO: a Hebrew BERT Model and a Tool for Polarity Analysis and Emotion Recognition [0.30458514384586394]
  HeBERT is a transformer-based model for modern Hebrew text. HebEMO is a tool that uses HeBERT to detect polarity and extract emotions from Hebrew user-generated content.
  arXiv  Detail & Related papers  (2021-02-03T06:59:59Z)
• Transformer-based Language Model Fine-tuning Methods for COVID-19 Fake News Detection [7.29381091750894]
  We propose a novel transformer-based language model fine-tuning approach for these fake news detection. First, the token vocabulary of individual model is expanded for the actual semantics of professional phrases. Last, the predicted features extracted by universal language model RoBERTa and domain-specific model CT-BERT are fused by one multiple layer perception to integrate fine-grained and high-level specific representations.
  arXiv  Detail & Related papers  (2021-01-14T09:05:42Z)
• Explicit Alignment Objectives for Multilingual Bidirectional Encoders [111.65322283420805]
  We present a new method for learning multilingual encoders, AMBER (Aligned Multilingual Bi-directional EncodeR) AMBER is trained on additional parallel data using two explicit alignment objectives that align the multilingual representations at different granularities. Experimental results show that AMBER obtains gains of up to 1.1 average F1 score on sequence tagging and up to 27.3 average accuracy on retrieval over the XLMR-large model.
  arXiv  Detail & Related papers  (2020-10-15T18:34:13Z)
• Pronoun-Targeted Fine-tuning for NMT with Hybrid Losses [6.596002578395152]
  We introduce a class of conditional generative-discriminative hybrid losses that we use to fine-tune a trained machine translation model. We improve the model performance of both a sentence-level and a contextual model without using any additional data. Our sentence-level model shows a 0.5 BLEU improvement on both the WMT14 and the IWSLT13 De-En testsets. Our contextual model achieves the best results, improving from 31.81 to 32 BLEU on WMT14 De-En testset, and from 32.10 to 33.13 on the IWSLT13 De-En
  arXiv  Detail & Related papers  (2020-10-15T10:11:40Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.