EmoBench-UA: A Benchmark Dataset for Emotion Detection in Ukrainian

• URL: http://arxiv.org/abs/2505.23297v1
• Date: Thu, 29 May 2025 09:49:57 GMT
• Title: EmoBench-UA: A Benchmark Dataset for Emotion Detection in Ukrainian
• Authors: Daryna Dementieva, Nikolay Babakov, Alexander Fraser,
• Abstract summary: EmoBench-UA is the first annotated dataset for emotion detection in Ukrainian texts.<n>Our findings highlight the challenges of emotion classification in non-mainstream languages like Ukrainian.
• Score: 60.61343989805093
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: While Ukrainian NLP has seen progress in many texts processing tasks, emotion classification remains an underexplored area with no publicly available benchmark to date. In this work, we introduce EmoBench-UA, the first annotated dataset for emotion detection in Ukrainian texts. Our annotation schema is adapted from the previous English-centric works on emotion detection (Mohammad et al., 2018; Mohammad, 2022) guidelines. The dataset was created through crowdsourcing using the Toloka.ai platform ensuring high-quality of the annotation process. Then, we evaluate a range of approaches on the collected dataset, starting from linguistic-based baselines, synthetic data translated from English, to large language models (LLMs). Our findings highlight the challenges of emotion classification in non-mainstream languages like Ukrainian and emphasize the need for further development of Ukrainian-specific models and training resources.

Related papers

• BRIGHTER: BRIdging the Gap in Human-Annotated Textual Emotion Recognition Datasets for 28 Languages [93.92804151830744]
  We present BRIGHTER, a collection of multi-labeled, emotion-annotated datasets in 28 different languages.<n>We highlight the challenges related to the data collection and annotation processes.<n>We show that the BRIGHTER datasets represent a meaningful step towards addressing the gap in text-based emotion recognition.
  arXiv  Detail & Related papers  (2025-02-17T15:39:50Z)
• MASIVE: Open-Ended Affective State Identification in English and Spanish [10.41502827362741]
  In this work, we broaden our scope to a practically unbounded set of textitaffective states, which includes any terms that humans use to describe their experiences of feeling. We collect and publish MASIVE, a dataset of Reddit posts in English and Spanish containing over 1,000 unique affective states each. On this task, we find that smaller finetuned multilingual models outperform much larger LLMs, even on region-specific Spanish affective states.
  arXiv  Detail & Related papers  (2024-07-16T21:43:47Z)
• Dataset and Benchmark for Urdu Natural Scenes Text Detection, Recognition and Visual Question Answering [50.52792174648067]
  This initiative seeks to bridge the gap between textual and visual comprehension. We propose a new multi-task Urdu scene text dataset comprising over 1000 natural scene images. We provide fine-grained annotations for text instances, addressing the limitations of previous datasets.
  arXiv  Detail & Related papers  (2024-05-21T06:48:26Z)
• SER_AMPEL: a multi-source dataset for speech emotion recognition of Italian older adults [58.49386651361823]
  SER_AMPEL is a multi-source dataset for speech emotion recognition (SER) It is collected with the aim of providing a reference for speech emotion recognition in case of Italian older adults. The evidence of the need for such a dataset emerges from the analysis of the state of the art.
  arXiv  Detail & Related papers  (2023-11-24T13:47:25Z)
• When a Language Question Is at Stake. A Revisited Approach to Label Sensitive Content [0.0]
  Article revisits an approach of pseudo-labeling sensitive data on the example of Ukrainian tweets covering the Russian-Ukrainian war. We provide a fundamental statistical analysis of the obtained data, evaluation of models used for pseudo-labelling, and set further guidelines on how the scientists can leverage the corpus.
  arXiv  Detail & Related papers  (2023-11-17T13:35:10Z)
• MSVD-Indonesian: A Benchmark for Multimodal Video-Text Tasks in Indonesian [0.0]
  We introduce the first public Indonesian video-text dataset by translating the English captions in the MSVD dataset into Indonesian.<n>We evaluate neural network models which were developed for the English video-text dataset on three tasks.<n>We apply a cross-lingual transfer learning approach by leveraging English-pretrained extractors and fine-tuning models on our Indonesian dataset.
  arXiv  Detail & Related papers  (2023-06-20T07:19:36Z)
• Leveraging Language Identification to Enhance Code-Mixed Text Classification [0.7340017786387767]
  Existing deep-learning models do not take advantage of the implicit language information in code-mixed text. Our study aims to improve BERT-based models performance on low-resource Code-Mixed Hindi-English datasets.
  arXiv  Detail & Related papers  (2023-06-08T06:43:10Z)
• Speech-to-Speech Translation For A Real-world Unwritten Language [62.414304258701804]
  We study speech-to-speech translation (S2ST) that translates speech from one language into another language. We present an end-to-end solution from training data collection, modeling choices to benchmark dataset release.
  arXiv  Detail & Related papers  (2022-11-11T20:21:38Z)
• Methods for Detoxification of Texts for the Russian Language [55.337471467610094]
  We introduce the first study of automatic detoxification of Russian texts to combat offensive language. We test two types of models - unsupervised approach that performs local corrections and supervised approach based on pretrained language GPT-2 model. The results show that the tested approaches can be successfully used for detoxification, although there is room for improvement.
  arXiv  Detail & Related papers  (2021-05-19T10:37:44Z)
• XED: A Multilingual Dataset for Sentiment Analysis and Emotion Detection [0.42056926734482064]
  The dataset consists of human-annotated Finnish (25k) and English sentences (30k) We use Plutchik's core emotions to annotate the dataset with the addition of neutral to create a multilabel multiclass dataset. The dataset is carefully evaluated using language-specific BERT models and SVMs to show that XED performs on par with other similar datasets.
  arXiv  Detail & Related papers  (2020-11-03T10:43:22Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.