BanglaNLG: Benchmarks and Resources for Evaluating Low-Resource Natural Language Generation in Bangla

• URL: http://arxiv.org/abs/2205.11081v2
• Date: Tue, 24 May 2022 01:33:34 GMT
• Title: BanglaNLG: Benchmarks and Resources for Evaluating Low-Resource Natural Language Generation in Bangla
• Authors: Abhik Bhattacharjee, Tahmid Hasan, Wasi Uddin Ahmad, Rifat Shahriyar
• Abstract summary: This work presents a benchmark for evaluating natural language generation models in Bangla. We aggregate three challenging conditional text generation tasks under the BanglaNLG benchmark. Using a clean corpus of 27.5 GB of Bangla data, we pretrain BanglaT5, a sequence-to-sequence Transformer model for Bangla. BanglaT5 achieves state-of-the-art performance in all of these tasks, outperforming mT5 (base) by up to 5.4%.
• Score: 21.47743471497797
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This work presents BanglaNLG, a comprehensive benchmark for evaluating natural language generation (NLG) models in Bangla, a widely spoken yet low-resource language in the web domain. We aggregate three challenging conditional text generation tasks under the BanglaNLG benchmark. Then, using a clean corpus of 27.5 GB of Bangla data, we pretrain BanglaT5, a sequence-to-sequence Transformer model for Bangla. BanglaT5 achieves state-of-the-art performance in all of these tasks, outperforming mT5 (base) by up to 5.4%. We are making the BanglaT5 language model and a leaderboard publicly available in the hope of advancing future research and evaluation on Bangla NLG. The resources can be found at https://github.com/csebuetnlp/BanglaNLG.

Related papers

• LowResource at BLP-2023 Task 2: Leveraging BanglaBert for Low Resource Sentiment Analysis of Bangla Language [0.5922488908114022]
  This paper describes the system of the LowResource Team for Task 2 of BLP-2023. It involves conducting sentiment analysis on a dataset composed of public posts and comments from diverse social media platforms. Our primary aim is to utilize BanglaBert, a BERT model pre-trained on a large Bangla corpus.
  arXiv  Detail & Related papers  (2023-11-21T17:21:15Z)
• On Evaluation of Bangla Word Analogies [0.8658596218544772]
  This paper presents a high-quality dataset for evaluating the quality of Bangla word embeddings. Despite being the 7th most-spoken language in the world, Bangla is a low-resource language and popular NLP models fail to perform well.
  arXiv  Detail & Related papers  (2023-04-10T14:27:35Z)
• BanglaParaphrase: A High-Quality Bangla Paraphrase Dataset [3.922582192616519]
  We present BanglaParaphrase, a high-quality synthetic Bangla Paraphrase dataset curated by a novel filtering pipeline. We aim to take a step towards alleviating the low resource status of the Bangla language in the NLP domain through the introduction of BanglaParaphrase.
  arXiv  Detail & Related papers  (2022-10-11T02:52:31Z)
• Evaluation of Transfer Learning for Polish with a Text-to-Text Model [54.81823151748415]
  We introduce a new benchmark for assessing the quality of text-to-text models for Polish. The benchmark consists of diverse tasks and datasets: KLEJ benchmark adapted for text-to-text, en-pl translation, summarization, and question answering. We present plT5 - a general-purpose text-to-text model for Polish that can be fine-tuned on various Natural Language Processing (NLP) tasks with a single training objective.
  arXiv  Detail & Related papers  (2022-05-18T09:17:14Z)
• End-to-End Natural Language Understanding Pipeline for Bangla Conversational Agents [0.43012765978447565]
  We propose a novel approach to build a business assistant which can communicate in Bangla and Bangla Transliteration in English with high confidence consistently. We use Rasa Open Source Framework, fastText embeddings, Polyglot embeddings, Flask, and other systems as building blocks. We present a pipeline for intent classification and entity extraction which achieves reasonable performance.
  arXiv  Detail & Related papers  (2021-07-12T16:09:22Z)
• A Review of Bangla Natural Language Processing Tasks and the Utility of Transformer Models [2.5768647103950357]
  We provide a review of Bangla NLP tasks, resources, and tools available to the research community. We benchmark datasets collected from various platforms for nine NLP tasks using current state-of-the-art algorithms. We report our results using both individual and consolidated datasets and provide data for future research.
  arXiv  Detail & Related papers  (2021-07-08T13:49:46Z)
• IndoNLG: Benchmark and Resources for Evaluating Indonesian Natural Language Generation [45.90242600586664]
  We introduce IndoNLG, the first such benchmark for the Indonesian language for natural language generation (NLG) We provide a vast and clean pre-training corpus of Indonesian, Sundanese, and Javanese datasets called Indo4B-Plus. We evaluate the effectiveness and efficiency of IndoBART by conducting extensive evaluation on all IndoNLG tasks.
  arXiv  Detail & Related papers  (2021-04-16T16:16:44Z)
• GLGE: A New General Language Generation Evaluation Benchmark [139.25515221280767]
  General Language Generation Evaluation (GLGE) is a new multi-task benchmark for evaluating the generalization capabilities of NLG models. To encourage research on pretraining and transfer learning on NLG models, we make GLGE publicly available and build a leaderboard with strong baselines.
  arXiv  Detail & Related papers  (2020-11-24T06:59:45Z)
• mT5: A massively multilingual pre-trained text-to-text transformer [60.0210636815514]
  "Text-to-Text Transfer Transformer" (T5) leveraged a unified text-to-text format and scale to attain state-of-the-art results on English-language NLP tasks. We introduce mT5, a multilingual variant of T5 that was pre-trained on a new Common Crawl-based dataset covering 101 languages.
  arXiv  Detail & Related papers  (2020-10-22T17:58:14Z)
• KGPT: Knowledge-Grounded Pre-Training for Data-to-Text Generation [100.79870384880333]
  We propose a knowledge-grounded pre-training (KGPT) to generate knowledge-enriched text. We adopt three settings, namely fully-supervised, zero-shot, few-shot to evaluate its effectiveness. Under zero-shot setting, our model achieves over 30 ROUGE-L on WebNLG while all other baselines fail.
  arXiv  Detail & Related papers  (2020-10-05T19:59:05Z)
• TuringAdvice: A Generative and Dynamic Evaluation of Language Use [90.3029315711237]
  We propose TuringAdvice, a new challenge task and dataset for language understanding models. Given a written situation that a real person is currently facing, a model must generate helpful advice in natural language. Empirical results show that today's models struggle at TuringAdvice.
  arXiv  Detail & Related papers  (2020-04-07T18:00:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.