L3Cube-MahaCorpus and MahaBERT: Marathi Monolingual Corpus, Marathi BERT Language Models, and Resources

• URL: http://arxiv.org/abs/2202.01159v1
• Date: Wed, 2 Feb 2022 17:35:52 GMT
• Title: L3Cube-MahaCorpus and MahaBERT: Marathi Monolingual Corpus, Marathi BERT Language Models, and Resources
• Authors: Raviraj Joshi
• Abstract summary: We present L3Cube-MahaCorpus a Marathi monolingual data set scraped from different internet sources. We expand the existing Marathi monolingual corpus with 24.8M sentences and 289M tokens. We show the effectiveness of these resources on downstream classification and NER tasks.
• Score: 1.14219428942199
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We present L3Cube-MahaCorpus a Marathi monolingual data set scraped from different internet sources. We expand the existing Marathi monolingual corpus with 24.8M sentences and 289M tokens. We further present, MahaBERT, MahaAlBERT, and MahaRoBerta all BERT-based masked language models, and MahaFT, the fast text word embeddings both trained on full Marathi corpus with 752M tokens. We show the effectiveness of these resources on downstream classification and NER tasks. Marathi is a popular language in India but still lacks these resources. This work is a step forward in building open resources for the Marathi language. The data and models are available at https://github.com/l3cube-pune/MarathiNLP .

Related papers

• Dictionary Insertion Prompting for Multilingual Reasoning on Multilingual Large Language Models [52.00446751692225]
  We present a novel and simple yet effective method called textbfDictionary textbfInsertion textbfPrompting (textbfDIP) When providing a non-English prompt, DIP looks up a word dictionary and inserts words' English counterparts into the prompt for LLMs. It then enables better translation into English and better English model thinking steps which leads to obviously better results.
  arXiv  Detail & Related papers  (2024-11-02T05:10:50Z)
• CoSTA: Code-Switched Speech Translation using Aligned Speech-Text Interleaving [61.73180469072787]
  We focus on the problem of spoken translation (ST) of code-switched speech in Indian languages to English text. We present a new end-to-end model architecture COSTA that scaffolds on pretrained automatic speech recognition (ASR) and machine translation (MT) modules. COSTA significantly outperforms many competitive cascaded and end-to-end multimodal baselines by up to 3.5 BLEU points.
  arXiv  Detail & Related papers  (2024-06-16T16:10:51Z)
• TransMI: A Framework to Create Strong Baselines from Multilingual Pretrained Language Models for Transliterated Data [50.40191599304911]
  We propose Transliterate transliteration-Merge (TransMI), which can create a strong baseline well-suited for data that is transliterated into a common script. Results show a consistent improvement of 3% to 34%, varying across different models and tasks.
  arXiv  Detail & Related papers  (2024-05-16T09:08:09Z)
• L3Cube-MahaNLP: Marathi Natural Language Processing Datasets, Models, and Library [1.14219428942199]
  Despite being the third most popular language in India, the Marathi language lacks useful NLP resources. With L3Cube-MahaNLP, we aim to build resources and a library for Marathi natural language processing. We present datasets and transformer models for supervised tasks like sentiment analysis, named entity recognition, and hate speech detection.
  arXiv  Detail & Related papers  (2022-05-29T17:51:00Z)
• Mono vs Multilingual BERT for Hate Speech Detection and Text Classification: A Case Study in Marathi [0.966840768820136]
  We focus on the Marathi language and evaluate the models on the datasets for hate speech detection, sentiment analysis and simple text classification in Marathi. We use standard multilingual models such as mBERT, indicBERT and xlm-RoBERTa and compare with MahaBERT, MahaALBERT and MahaRoBERTa, the monolingual models for Marathi. We show that monolingual MahaBERT based models provide rich representations as compared to sentence embeddings from multi-lingual counterparts.
  arXiv  Detail & Related papers  (2022-04-19T05:07:58Z)
• L3Cube-MahaNER: A Marathi Named Entity Recognition Dataset and BERT models [0.7874708385247353]
  We focus on Marathi, an Indian language, spoken prominently by the people of Maharashtra state. We present L3Cube-MahaNER, the first major gold standard named entity recognition dataset in Marathi. In the end, we benchmark the dataset on different CNN, LSTM, and Transformer based models like mBERT, XLM-RoBERTa, IndicBERT, MahaBERT, etc.
  arXiv  Detail & Related papers  (2022-04-12T18:32:15Z)
• "A Passage to India": Pre-trained Word Embeddings for Indian Languages [30.607474624873014]
  We use various existing approaches to create multiple word embeddings for 14 Indian languages. We place these embeddings for all these languages in a single repository. We release a total of 436 models using 8 different approaches.
  arXiv  Detail & Related papers  (2021-12-27T17:31:04Z)
• Experimental Evaluation of Deep Learning models for Marathi Text Classification [0.0]
  We evaluate CNN, LSTM, ULMFiT, and BERT based models on two publicly available Marathi text classification datasets. We show that basic single layer models based on CNN and LSTM coupled with FastText embeddings perform on par with the BERT based models on the available datasets.
  arXiv  Detail & Related papers  (2021-01-13T06:21:27Z)
• UNKs Everywhere: Adapting Multilingual Language Models to New Scripts [103.79021395138423]
  Massively multilingual language models such as multilingual BERT (mBERT) and XLM-R offer state-of-the-art cross-lingual transfer performance on a range of NLP tasks. Due to their limited capacity and large differences in pretraining data, there is a profound performance gap between resource-rich and resource-poor target languages. We propose novel data-efficient methods that enable quick and effective adaptation of pretrained multilingual models to such low-resource languages and unseen scripts.
  arXiv  Detail & Related papers  (2020-12-31T11:37:28Z)
• Anchor-based Bilingual Word Embeddings for Low-Resource Languages [76.48625630211943]
  Good quality monolingual word embeddings (MWEs) can be built for languages which have large amounts of unlabeled text. MWEs can be aligned to bilingual spaces using only a few thousand word translation pairs. This paper proposes a new approach for building BWEs in which the vector space of the high resource source language is used as a starting point.
  arXiv  Detail & Related papers  (2020-10-23T19:17:00Z)
• A Multilingual Parallel Corpora Collection Effort for Indian Languages [43.62422999765863]
  We present sentence aligned parallel corpora across 10 Indian languages - Hindi, Telugu, Tamil, Malayalam, Gujarati, Urdu, Bengali, Oriya, Marathi, Punjabi, and English. The corpora are compiled from online sources which have content shared across languages.
  arXiv  Detail & Related papers  (2020-07-15T14:00:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.