CEAID: Benchmark of Multilingual Machine-Generated Text Detection Methods for Central European Languages

• URL: http://arxiv.org/abs/2509.26051v1
• Date: Tue, 30 Sep 2025 10:27:53 GMT
• Title: CEAID: Benchmark of Multilingual Machine-Generated Text Detection Methods for Central European Languages
• Authors: Dominik Macko, Jakub Kopal,
• Abstract summary: We provide the first benchmark of detection methods focused on Central European languages.<n>We compare train-languages combinations to identify the best performing ones.<n>Supervised finetuned detectors in the Central European languages are found the most performant in these languages.
• Score: 4.089936423985361
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine-generated text detection, as an important task, is predominantly focused on English in research. This makes the existing detectors almost unusable for non-English languages, relying purely on cross-lingual transferability. There exist only a few works focused on any of Central European languages, leaving the transferability towards these languages rather unexplored. We fill this gap by providing the first benchmark of detection methods focused on this region, while also providing comparison of train-languages combinations to identify the best performing ones. We focus on multi-domain, multi-generator, and multilingual evaluation, pinpointing the differences of individual aspects, as well as adversarial robustness of detection methods. Supervised finetuned detectors in the Central European languages are found the most performant in these languages as well as the most resistant against obfuscation.

Related papers

• MultiConAD: A Unified Multilingual Conversational Dataset for Early Alzheimer's Detection [12.803369138301163]
  We introduce a novel, multilingual dataset for AD detection by unifying 16 publicly available dementia-related conversational datasets.<n>Second, we perform finer-grained classification, including MCI, and evaluate various classifiers using sparse and dense text representations.<n>Third, we conduct experiments in monolingual and multilingual settings, finding that some languages benefit from multilingual training while others perform better independently.
  arXiv  Detail & Related papers  (2025-02-26T15:12:37Z)
• Automatic Discrimination of Human and Neural Machine Translation in Multilingual Scenarios [4.631167282648452]
  We tackle the task of automatically discriminating between human and machine translations. We perform experiments in a multilingual setting, considering multiple languages and multilingual pretrained language models.
  arXiv  Detail & Related papers  (2023-05-31T11:41:24Z)
• Revisiting Machine Translation for Cross-lingual Classification [91.43729067874503]
  Most research in the area focuses on the multilingual models rather than the Machine Translation component. We show that, by using a stronger MT system and mitigating the mismatch between training on original text and running inference on machine translated text, translate-test can do substantially better than previously assumed.
  arXiv  Detail & Related papers  (2023-05-23T16:56:10Z)
• Detecting Lexical Borrowings from Dominant Languages in Multilingual Wordlists [3.096615629099617]
  We test new methods for lexical borrowing detection in contact situations where dominant languages play an important role. All methods perform well, with the supervised machine learning system outperforming the classical systems. A review of detection errors shows that borrowing detection could be substantially improved by taking into account donor words with divergent meanings from recipient words.
  arXiv  Detail & Related papers  (2023-02-01T02:44:28Z)
• Analyzing the Mono- and Cross-Lingual Pretraining Dynamics of Multilingual Language Models [73.11488464916668]
  This study investigates the dynamics of the multilingual pretraining process. We probe checkpoints taken from throughout XLM-R pretraining, using a suite of linguistic tasks. Our analysis shows that the model achieves high in-language performance early on, with lower-level linguistic skills acquired before more complex ones.
  arXiv  Detail & Related papers  (2022-05-24T03:35:00Z)
• On Cross-Lingual Retrieval with Multilingual Text Encoders [51.60862829942932]
  We study the suitability of state-of-the-art multilingual encoders for cross-lingual document and sentence retrieval tasks. We benchmark their performance in unsupervised ad-hoc sentence- and document-level CLIR experiments. We evaluate multilingual encoders fine-tuned in a supervised fashion (i.e., we learn to rank) on English relevance data in a series of zero-shot language and domain transfer CLIR experiments.
  arXiv  Detail & Related papers  (2021-12-21T08:10:27Z)
• It's All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoning [4.200736775540874]
  We design a simple approach to commonsense reasoning which trains a linear classifier with weights of multi-head attention as features. The method performs competitively with recent supervised and unsupervised approaches for commonsense reasoning. Most of the performance is given by the same small subset of attention heads for all studied languages.
  arXiv  Detail & Related papers  (2021-06-22T21:25:43Z)
• AM2iCo: Evaluating Word Meaning in Context across Low-ResourceLanguages with Adversarial Examples [51.048234591165155]
  We present AM2iCo, Adversarial and Multilingual Meaning in Context. It aims to faithfully assess the ability of state-of-the-art (SotA) representation models to understand the identity of word meaning in cross-lingual contexts. Results reveal that current SotA pretrained encoders substantially lag behind human performance.
  arXiv  Detail & Related papers  (2021-04-17T20:23:45Z)
• Multilingual and Cross-Lingual Intent Detection from Spoken Data [36.116844659291885]
  MInDS-14 is a first training and evaluation resource for the intent detection task with spoken data. Our results indicate that combining machine translation models with state-of-the-art multilingual sentence encoders can yield strong intent detectors. We see this work as an important step towards more inclusive development and evaluation of multilingual intent detectors from spoken data.
  arXiv  Detail & Related papers  (2021-04-17T12:17:28Z)
• Evaluating Multilingual Text Encoders for Unsupervised Cross-Lingual Retrieval [51.60862829942932]
  We present a systematic empirical study focused on the suitability of the state-of-the-art multilingual encoders for cross-lingual document and sentence retrieval tasks. For sentence-level CLIR, we demonstrate that state-of-the-art performance can be achieved. However, the peak performance is not met using the general-purpose multilingual text encoders off-the-shelf', but rather relying on their variants that have been further specialized for sentence understanding tasks.
  arXiv  Detail & Related papers  (2021-01-21T00:15:38Z)
• XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization [128.37244072182506]
  Cross-lingual TRansfer Evaluation of Multilinguals XTREME is a benchmark for evaluating the cross-lingual generalization capabilities of multilingual representations across 40 languages and 9 tasks. We demonstrate that while models tested on English reach human performance on many tasks, there is still a sizable gap in the performance of cross-lingually transferred models.
  arXiv  Detail & Related papers  (2020-03-24T19:09:37Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.