Neural Grammatical Error Correction for Romanian

• URL: http://arxiv.org/abs/2604.23627v1
• Date: Sun, 26 Apr 2026 09:42:49 GMT
• Title: Neural Grammatical Error Correction for Romanian
• Abstract summary: We introduce a first Grammatical Error Correction corpus for Romanian consisting of 10k pairs of sentences.<n>Multiple neural models were experimented, together with pretraining strategies, which proved effective for GEC in low-resource settings.
• Score: 1.2680800636608986
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Resources for Grammatical Error Correction (GEC) in non-English languages are scarce, while available spellcheckers in these languages are mostly limited to simple corrections and rules. In this paper we introduce a first GEC corpus for Romanian consisting of 10k pairs of sentences. In addition, the German version of ERRANT (ERRor ANnotation Toolkit) scorer was adapted for Romanian to analyze this corpus and extract edits needed for evaluation. Multiple neural models were experimented, together with pretraining strategies, which proved effective for GEC in low-resource settings. Our baseline consists of a small Transformer model trained only on the GEC dataset (F0.5 of 44.38), whereas the best performing model is produced by pretraining a larger Transformer model on artificially generated data, followed by finetuning on the actual corpus (F0.5 of 53.76). The proposed method for generating additional training examples is easily extensible and can be applied to any language, as it requires only a POS tagger

Related papers

• Encode Errors: Representational Retrieval of In-Context Demonstrations for Multilingual Grammatical Error Correction [28.920985595976873]
  Grammatical Error Correction (GEC) involves detecting and correcting the wrong usage of grammar.<n>Large language models (LLMs) with in-context learning (ICL) capabilities have shown significant progress on various natural language processing (NLP) tasks.<n>This is mainly due to the challenge of retrieving suitable in-context demonstrations that capture error patterns instead of semantic similarity.
  arXiv  Detail & Related papers  (2026-06-13T18:00:26Z)
• Contextual Spelling Correction with Language Model for Low-resource Setting [0.0]
  A small-scale word-based transformer LM is trained to provide the SC model with contextual understanding. Probability of error happening(error model) is extracted from the corpus. Combination of LM and error model is used to develop the SC model through the well-known noisy channel framework.
  arXiv  Detail & Related papers  (2024-04-28T05:29:35Z)
• GECTurk: Grammatical Error Correction and Detection Dataset for Turkish [1.804922416527064]
  Grammatical Error Detection and Correction (GEC) tools have proven useful for native speakers and second language learners. Synthetic data generation is a common practice to overcome the scarcity of such data. We present a flexible and synthetic data generation pipeline for Turkish covering more than 20 expert-curated grammar and spelling rules.
  arXiv  Detail & Related papers  (2023-09-20T14:25:44Z)
• Ensemble Transfer Learning for Multilingual Coreference Resolution [60.409789753164944]
  A problem that frequently occurs when working with a non-English language is the scarcity of annotated training data. We design a simple but effective ensemble-based framework that combines various transfer learning techniques. We also propose a low-cost TL method that bootstraps coreference resolution models by utilizing Wikipedia anchor texts.
  arXiv  Detail & Related papers  (2023-01-22T18:22:55Z)
• A Unified Strategy for Multilingual Grammatical Error Correction with Pre-trained Cross-Lingual Language Model [100.67378875773495]
  We propose a generic and language-independent strategy for multilingual Grammatical Error Correction. Our approach creates diverse parallel GEC data without any language-specific operations. It achieves the state-of-the-art results on the NLPCC 2018 Task 2 dataset (Chinese) and obtains competitive performance on Falko-Merlin (German) and RULEC-GEC (Russian)
  arXiv  Detail & Related papers  (2022-01-26T02:10:32Z)
• Factorized Neural Transducer for Efficient Language Model Adaptation [51.81097243306204]
  We propose a novel model, factorized neural Transducer, by factorizing the blank and vocabulary prediction. It is expected that this factorization can transfer the improvement of the standalone language model to the Transducer for speech recognition. We demonstrate that the proposed factorized neural Transducer yields 15% to 20% WER improvements when out-of-domain text data is used for language model adaptation.
  arXiv  Detail & Related papers  (2021-09-27T15:04:00Z)
• Sentence Bottleneck Autoencoders from Transformer Language Models [53.350633961266375]
  We build a sentence-level autoencoder from a pretrained, frozen transformer language model. We adapt the masked language modeling objective as a generative, denoising one, while only training a sentence bottleneck and a single-layer modified transformer decoder. We demonstrate that the sentence representations discovered by our model achieve better quality than previous methods that extract representations from pretrained transformers on text similarity tasks, style transfer, and single-sentence classification tasks in the GLUE benchmark, while using fewer parameters than large pretrained models.
  arXiv  Detail & Related papers  (2021-08-31T19:39:55Z)
• A Simple Recipe for Multilingual Grammatical Error Correction [6.262434757334487]
  This paper presents a recipe to train state-of-the-art multilingual Grammatical Error Correction (GEC) models. We first propose a language-agnostic method to generate a large number of synthetic examples. The second ingredient is to use large-scale multilingual language models.
  arXiv  Detail & Related papers  (2021-06-07T17:47:04Z)
• Rejuvenating Low-Frequency Words: Making the Most of Parallel Data in Non-Autoregressive Translation [98.11249019844281]
  Knowledge distillation (KD) is commonly used to construct synthetic data for training non-autoregressive translation (NAT) models. We propose reverse KD to rejuvenate more alignments for low-frequency target words. Results demonstrate that the proposed approach can significantly and universally improve translation quality.
  arXiv  Detail & Related papers  (2021-06-02T02:41:40Z)
• The birth of Romanian BERT [1.377045689881944]
  This paper introduces Romanian BERT, the first purely Romanian transformer-based language model, pretrained on a large text corpus. We discuss corpus composition and cleaning, the model training process, as well as an extensive evaluation of the model on various Romanian datasets.
  arXiv  Detail & Related papers  (2020-09-18T09:30:48Z)
• Stronger Baselines for Grammatical Error Correction Using Pretrained Encoder-Decoder Model [24.51571980021599]
  We explore the utility of bidirectional and auto-regressive transformers (BART) as a generic pretrained encoder-decoder model for grammatical error correction (GEC) We find that monolingual and multilingual BART models achieve high performance in GEC, with one of the results being comparable to the current strong results in English GEC.
  arXiv  Detail & Related papers  (2020-05-24T22:13:24Z)
• Parameter Space Factorization for Zero-Shot Learning across Tasks and Languages [112.65994041398481]
  We propose a Bayesian generative model for the space of neural parameters. We infer the posteriors over such latent variables based on data from seen task-language combinations. Our model yields comparable or better results than state-of-the-art, zero-shot cross-lingual transfer methods.
  arXiv  Detail & Related papers  (2020-01-30T16:58:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.