PORTULAN ExtraGLUE Datasets and Models: Kick-starting a Benchmark for the Neural Processing of Portuguese

• URL: http://arxiv.org/abs/2404.05333v3
• Date: Wed, 8 May 2024 19:32:42 GMT
• Title: PORTULAN ExtraGLUE Datasets and Models: Kick-starting a Benchmark for the Neural Processing of Portuguese
• Authors: Tomás Osório, Bernardo Leite, Henrique Lopes Cardoso, Luís Gomes, João Rodrigues, Rodrigo Santos, António Branco,
• Abstract summary: We contribute a collection of datasets for an array of language processing tasks and a collection of fine-tuned neural language models on these downstream tasks. To align with mainstream benchmarks in the literature, originally developed in English, the datasets were machine-translated from English with a state-of-the-art translation engine. The resulting PORTULAN ExtraGLUE benchmark is a basis for research on Portuguese whose improvement can be pursued in future work.
• Score: 1.2779732438508473
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Leveraging research on the neural modelling of Portuguese, we contribute a collection of datasets for an array of language processing tasks and a corresponding collection of fine-tuned neural language models on these downstream tasks. To align with mainstream benchmarks in the literature, originally developed in English, and to kick start their Portuguese counterparts, the datasets were machine-translated from English with a state-of-the-art translation engine. The resulting PORTULAN ExtraGLUE benchmark is a basis for research on Portuguese whose improvement can be pursued in future work. Similarly, the respective fine-tuned neural language models, developed with a low-rank adaptation approach, are made available as baselines that can stimulate future work on the neural processing of Portuguese. All datasets and models have been developed and are made available for two variants of Portuguese: European and Brazilian.

Related papers

• Enhancing Portuguese Variety Identification with Cross-Domain Approaches [2.31011809034817]
  We develop a cross-domain language variety identifier (LVI) to discriminate between European and Brazilian Portuguese. Although this research focuses on two Portuguese varieties, our contribution can be extended to other varieties and languages.
  arXiv  Detail & Related papers  (2025-02-20T09:31:48Z)
• Tradutor: Building a Variety Specific Translation Model [3.976102757693942]
  We introduce the first open-source translation model specifically tailored for European Portuguese. Our best model surpasses existing open-source translation systems for Portuguese. By making our dataset, models, and code publicly available, we aim to support and encourage further research.
  arXiv  Detail & Related papers  (2025-02-20T09:20:59Z)
• Tucano: Advancing Neural Text Generation for Portuguese [0.0]
  This study aims to introduce a new set of resources to stimulate the future development of neural text generation in Portuguese. In this work, we document the development of GigaVerbo, a concatenation of deduplicated Portuguese text corpora amounting to 200 billion tokens. Our models perform equal or superior to other Portuguese and multilingual language models of similar size in several Portuguese benchmarks.
  arXiv  Detail & Related papers  (2024-11-12T15:06:06Z)
• From Brazilian Portuguese to European Portuguese [2.048226951354646]
  Brazilian Portuguese and European Portuguese are two varieties of the same language. There is a significant disproportion in the availability of resources between the two variants. This inequity can impact the quality of translation services accessible to European Portuguese speakers.
  arXiv  Detail & Related papers  (2024-08-14T10:58:48Z)
• A Novel Cartography-Based Curriculum Learning Method Applied on RoNLI: The First Romanian Natural Language Inference Corpus [71.77214818319054]
  Natural language inference is a proxy for natural language understanding. There is no publicly available NLI corpus for the Romanian language. We introduce the first Romanian NLI corpus (RoNLI) comprising 58K training sentence pairs.
  arXiv  Detail & Related papers  (2024-05-20T08:41:15Z)
• Natural Language Processing for Dialects of a Language: A Survey [56.93337350526933]
  State-of-the-art natural language processing (NLP) models are trained on massive training corpora, and report a superlative performance on evaluation datasets. This survey delves into an important attribute of these datasets: the dialect of a language. Motivated by the performance degradation of NLP models for dialectal datasets and its implications for the equity of language technologies, we survey past research in NLP for dialects in terms of datasets, and approaches.
  arXiv  Detail & Related papers  (2024-01-11T03:04:38Z)
• Multi-lingual Evaluation of Code Generation Models [82.7357812992118]
  We present new benchmarks on evaluation code generation models: MBXP and Multilingual HumanEval, and MathQA-X. These datasets cover over 10 programming languages. We are able to assess the performance of code generation models in a multi-lingual fashion.
  arXiv  Detail & Related papers  (2022-10-26T17:17:06Z)
• Summarize and Generate to Back-translate: Unsupervised Translation of Programming Languages [86.08359401867577]
  Back-translation is widely known for its effectiveness for neural machine translation when little to no parallel data is available. We propose performing back-translation via code summarization and generation. We show that our proposed approach performs competitively with state-of-the-art methods.
  arXiv  Detail & Related papers  (2022-05-23T08:20:41Z)
• Transformers and Transfer Learning for Improving Portuguese Semantic Role Labeling [2.9005223064604078]
  For low resource languages, and in particular for Portuguese, currently available SRL models are hindered by scarce training data. We explore a model architecture with only a pre-trained BERT-based model, a linear layer, softmax and Viterbi decoding.
  arXiv  Detail & Related papers  (2021-01-04T19:56:01Z)
• Pre-training Multilingual Neural Machine Translation by Leveraging Alignment Information [72.2412707779571]
  mRASP is an approach to pre-train a universal multilingual neural machine translation model. We carry out experiments on 42 translation directions across a diverse setting, including low, medium, rich resource, and as well as transferring to exotic language pairs.
  arXiv  Detail & Related papers  (2020-10-07T03:57:54Z)
• XCOPA: A Multilingual Dataset for Causal Commonsense Reasoning [68.57658225995966]
  Cross-lingual Choice of Plausible Alternatives (XCOPA) is a typologically diverse multilingual dataset for causal commonsense reasoning in 11 languages. We evaluate a range of state-of-the-art models on this novel dataset, revealing that the performance of current methods falls short compared to translation-based transfer.
  arXiv  Detail & Related papers  (2020-05-01T12:22:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.