Anubhuti -- An annotated dataset for emotional analysis of Bengali short stories

• URL: http://arxiv.org/abs/2010.03065v1
• Date: Tue, 6 Oct 2020 22:33:58 GMT
• Title: Anubhuti -- An annotated dataset for emotional analysis of Bengali short stories
• Authors: Aditya Pal and Bhaskar Karn
• Abstract summary: Anubhuti is the first and largest text corpus for analyzing emotions expressed by writers of Bengali short stories. We explain the data collection methods, the manual annotation process and the resulting high inter-annotator agreement. We have verified the performance of our dataset with baseline Machine Learning and a Deep Learning model for emotion classification.
• Score: 2.3424047967193826
• License: http://creativecommons.org/publicdomain/zero/1.0/
• Abstract: Thousands of short stories and articles are being written in many different languages all around the world today. Bengali, or Bangla, is the second highest spoken language in India after Hindi and is the national language of the country of Bangladesh. This work reports in detail the creation of Anubhuti -- the first and largest text corpus for analyzing emotions expressed by writers of Bengali short stories. We explain the data collection methods, the manual annotation process and the resulting high inter-annotator agreement of the dataset due to the linguistic expertise of the annotators and the clear methodology of labelling followed. We also address some of the challenges faced in the collection of raw data and annotation process of a low resource language like Bengali. We have verified the performance of our dataset with baseline Machine Learning as well as a Deep Learning model for emotion classification and have found that these standard models have a high accuracy and relevant feature selection on Anubhuti. In addition, we also explain how this dataset can be of interest to linguists and data analysts to study the flow of emotions as expressed by writers of Bengali literature.

Related papers

• L3Cube-MahaSum: A Comprehensive Dataset and BART Models for Abstractive Text Summarization in Marathi [0.4194295877935868]
  We present the MahaSUM dataset, a large-scale collection of diverse news articles in Marathi. The dataset was created by scraping articles from a wide range of online news sources and manually verifying the abstract summaries. We train an IndicBART model, a variant of the BART model tailored for Indic languages, using the MahaSUM dataset.
  arXiv  Detail & Related papers  (2024-10-11T18:37:37Z)
• Navigating Text-to-Image Generative Bias across Indic Languages [53.92640848303192]
  This research investigates biases in text-to-image (TTI) models for the Indic languages widely spoken across India. It evaluates and compares the generative performance and cultural relevance of leading TTI models in these languages against their performance in English.
  arXiv  Detail & Related papers  (2024-08-01T04:56:13Z)
• Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research [139.69207791947738]
  Dolma is a three-trillion-token English corpus built from a diverse mixture of web content, scientific papers, code, public-domain books, social media, and encyclopedic materials. We document Dolma, including its design principles, details about its construction, and a summary of its contents. We present analyses and experimental results on intermediate states of Dolma to share what we have learned about important data curation practices.
  arXiv  Detail & Related papers  (2024-01-31T20:29:50Z)
• Mukhyansh: A Headline Generation Dataset for Indic Languages [4.583536403673757]
  Mukhyansh is an extensive multilingual dataset, tailored for Indian language headline generation. Comprising over 3.39 million article-headline pairs, Mukhyansh spans across eight prominent Indian languages. Mukhyansh outperforms all other models, achieving an average ROUGE-L score of 31.43 across all 8 languages.
  arXiv  Detail & Related papers  (2023-11-29T15:49:24Z)
• NusaWrites: Constructing High-Quality Corpora for Underrepresented and Extremely Low-Resource Languages [54.808217147579036]
  We conduct a case study on Indonesian local languages. We compare the effectiveness of online scraping, human translation, and paragraph writing by native speakers in constructing datasets. Our findings demonstrate that datasets generated through paragraph writing by native speakers exhibit superior quality in terms of lexical diversity and cultural content.
  arXiv  Detail & Related papers  (2023-09-19T14:42:33Z)
• Cross-Lingual NER for Financial Transaction Data in Low-Resource Languages [70.25418443146435]
  We propose an efficient modeling framework for cross-lingual named entity recognition in semi-structured text data. We employ two independent datasets of SMSs in English and Arabic, each carrying semi-structured banking transaction information. With access to only 30 labeled samples, our model can generalize the recognition of merchants, amounts, and other fields from English to Arabic.
  arXiv  Detail & Related papers  (2023-07-16T00:45:42Z)
• Challenge Dataset of Cognates and False Friend Pairs from Indian Languages [54.6340870873525]
  Cognates are present in multiple variants of the same text across different languages. In this paper, we describe the creation of two cognate datasets for twelve Indian languages.
  arXiv  Detail & Related papers  (2021-12-17T14:23:43Z)
• Cross-lingual Offensive Language Identification for Low Resource Languages: The Case of Marathi [2.4737119633827174]
  MOLD is the first dataset of its kind compiled for Marathi, opening a new domain for research in low-resource Indo-Aryan languages. We present results from several machine learning experiments on this dataset, including zero-short and other transfer learning experiments on state-of-the-art cross-lingual transformers.
  arXiv  Detail & Related papers  (2021-09-08T11:29:44Z)
• HinFlair: pre-trained contextual string embeddings for pos tagging and text classification in the Hindi language [0.0]
  HinFlair is a language representation model (contextual string embeddings) pre-trained on a large monolingual Hindi corpus. Results show that HinFlair outperforms previous state-of-the-art publicly available pre-trained embeddings for downstream tasks like text classification and pos tagging.
  arXiv  Detail & Related papers  (2021-01-18T09:23:35Z)
• Sentiment analysis in Bengali via transfer learning using multi-lingual BERT [0.9883261192383611]
  In this paper, we present manually tagged 2-class and 3-class SA datasets in Bengali. We also demonstrate that the multi-lingual BERT model with relevant extensions can be trained via the approach of transfer learning. This deep learning model achieves an accuracy of 71% for 2-class sentiment classification compared to the current state-of-the-art accuracy of 68%.
  arXiv  Detail & Related papers  (2020-12-03T10:21:11Z)
• Comparison of Interactive Knowledge Base Spelling Correction Models for Low-Resource Languages [81.90356787324481]
  Spelling normalization for low resource languages is a challenging task because the patterns are hard to predict. This work shows a comparison of a neural model and character language models with varying amounts on target language data. Our usage scenario is interactive correction with nearly zero amounts of training examples, improving models as more data is collected.
  arXiv  Detail & Related papers  (2020-10-20T17:31:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.