ASCAT: An Arabic Scientific Corpus and Benchmark for Advanced Translation Evaluation

• URL: http://arxiv.org/abs/2604.00015v1
• Date: Tue, 10 Mar 2026 18:41:52 GMT
• Title: ASCAT: An Arabic Scientific Corpus and Benchmark for Advanced Translation Evaluation
• Abstract summary: ASCAT targets full scientific abstracts averaging 141.7 words (English) and 111.78 words (Arabic)<n>Each abstract was translated using three complementary generative AI (Gemini), transformer-based models (Hugging Face texttquickmt-en-ar), and commercial MT APIs (Google Translate, DeepL)<n>The resulting corpus contains 67,293 English tokens and 60,026 Arabic tokens, with an Arabic vocabulary of 17,604 unique words reflecting the morphological richness of the language.
• Score: 1.730057408524355
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: We present ASCAT (Arabic Scientific Corpus for Advanced Translation), a high-quality English-Arabic parallel benchmark corpus designed for scientific translation evaluation constructed through a systematic multi-engine translation and human validation pipeline. Unlike existing Arabic-English corpora that rely on short sentences or single-domain text, ASCAT targets full scientific abstracts averaging 141.7 words (English) and 111.78 words (Arabic), drawn from five scientific domains: physics, mathematics, computer science, quantum mechanics, and artificial intelligence. Each abstract was translated using three complementary architectures generative AI (Gemini), transformer-based models (Hugging Face \texttt{quickmt-en-ar}), and commercial MT APIs (Google Translate, DeepL) and subsequently validated by domain experts at the lexical, syntactic, and semantic levels. The resulting corpus contains 67,293 English tokens and 60,026 Arabic tokens, with an Arabic vocabulary of 17,604 unique words reflecting the morphological richness of the language. We benchmark three state-of-the-art LLMs on the corpus GPT-4o-mini (BLEU: 37.07), Gemini-3.0-Flash-Preview (BLEU: 30.44), and Qwen3-235B-A22B (BLEU: 23.68) demonstrating its discriminative power as an evaluation benchmark. ASCAT addresses a critical gap in scientific MT resources for Arabic and is designed to support rigorous evaluation of scientific translation quality and training of domain-specific translation models.

Related papers

• AraSSM: A bidirectional state-space encoder for Arabic masked language modeling [0.19921702137386235]
  AraSSM is a bidirectional Mamba encoder pretrained via masked language modeling on a corpus combining Arabic Wikipedia and CulturaX text.<n>We introduce AraSSM, a bidirectional Mamba encoder pretrained via masked language modeling on a corpus combining Arabic Wikipedia and CulturaX text.
  arXiv  Detail & Related papers  (2026-08-08T17:33:41Z)
• Bridging Scientific Heritage: An Arabic--Russian Parallel Corpus and LLM Benchmark for Sustainable Knowledge Transfer [0.0]
  We present a benchmark for Arabic--Russian scientific translation.<n>The benchmark includes a hybrid parallel corpus of about 27,000 sentence pairs.<n>It enables knowledge exchange between Arabic-speaking and Russian-speaking researchers.
  arXiv  Detail & Related papers  (2026-06-29T21:53:34Z)
• AfriScience-MT: Towards Decolonizing Science in Africa through Text Translation [13.80926536385416]
  AfriScience-MT is a parallel corpus covering six African languages (Amharic, Hausa, Luganda, Northern Sotho, Yorb, and isiZulu)<n>Professional translators translated plain-language summaries of scientific papers into each target language and created new terms where none existed.<n>We benchmark machine translation systems and large language models in zero-shot, few-shot, and fine-tuned settings.
  arXiv  Detail & Related papers  (2026-05-28T10:36:32Z)
• Character-Level Transformer for Tajik-Persian Transliteration with a Parallel Lexical Corpus [0.0]
  This study addresses automatic transliteration from Tajik (Cyrillic script) to Persian (Perso-Arabic script)<n>We present a curated, lexicographically verified parallel corpus of 52,152 Tajik--Persian words and short phrases.
  arXiv  Detail & Related papers  (2026-05-09T17:57:29Z)
• Omnilingual SONAR: Cross-Lingual and Cross-Modal Sentence Embeddings Bridging Massively Multilingual Text and Speech [61.759910921200834]
  Cross-lingual sentence encoders typically cover only a few hundred languages.<n>We introduce OmniSONAR, a new family of omnilingual, cross-lingual and cross-language sentence embedding models.
  arXiv  Detail & Related papers  (2026-03-17T14:47:35Z)
• Developing an English-Efik Corpus and Machine Translation System for Digitization Inclusion [0.0]
  This study evaluates the effectiveness of state-of-the-art multilingual neural machine translation models for English-Efik translation.<n>We fine-tuned both the mT5 multilingual model and the NLLB200 model on this dataset.<n>Our findings demonstrate the feasibility of developing practical machine translation tools for low-resource languages.
  arXiv  Detail & Related papers  (2026-03-16T06:17:22Z)
• AraLingBench A Human-Annotated Benchmark for Evaluating Arabic Linguistic Capabilities of Large Language Models [37.79823471716066]
  AraLingBench is a fully human annotated benchmark for evaluating the Arabic linguistic competence of large language models (LLMs)<n>The benchmark spans five core categories: grammar, morphology, spelling, reading comprehension, and syntax, through 150 expert-designed multiple choice questions.
  arXiv  Detail & Related papers  (2025-11-18T09:47:01Z)
• DialectalArabicMMLU: Benchmarking Dialectal Capabilities in Arabic and Multilingual Language Models [54.10223256792762]
  We present DialectalArabicMMLU, a new benchmark for evaluating the performance of large language models (LLMs) across Arabic dialects.<n>We extend the MMLU-Redux framework through manual translation and adaptation of 3K multiple-choice question-answer pairs into five major dialects.
  arXiv  Detail & Related papers  (2025-10-31T15:17:06Z)
• Science Across Languages: Assessing LLM Multilingual Translation of Scientific Papers [27.06887024646089]
  We leverage large language models (LLMs) to translate scientific articles.<n>We translate articles across multiple scientific disciplines into 28 languages.<n>Our benchmark results show an average performance of 95.9%.
  arXiv  Detail & Related papers  (2025-02-25T06:08:48Z)
• Artificial intelligence contribution to translation industry: looking back and forward [0.0]
  This study provides a comprehensive analysis of artificial intelligence (AI) contribution to research in the translation industry (ACTI) over forty-five years from 1980-2024.<n>13220 articles were retrieved from three sources, namely WoS, Scopus, and Lens; 9836 were unique records, which were used for the analysis.
  arXiv  Detail & Related papers  (2024-11-29T17:10:33Z)
• ArabicMMLU: Assessing Massive Multitask Language Understanding in Arabic [51.922112625469836]
  We present datasetname, the first multi-task language understanding benchmark for the Arabic language. Our data comprises 40 tasks and 14,575 multiple-choice questions in Modern Standard Arabic (MSA) and is carefully constructed by collaborating with native speakers in the region. Our evaluations of 35 models reveal substantial room for improvement, particularly among the best open-source models.
  arXiv  Detail & Related papers  (2024-02-20T09:07:41Z)
• CLSE: Corpus of Linguistically Significant Entities [58.29901964387952]
  We release a Corpus of Linguistically Significant Entities (CLSE) annotated by experts. CLSE covers 74 different semantic types to support various applications from airline ticketing to video games. We create a linguistically representative NLG evaluation benchmark in three languages: French, Marathi, and Russian.
  arXiv  Detail & Related papers  (2022-11-04T12:56:12Z)
• A Massively Multilingual Analysis of Cross-linguality in Shared Embedding Space [61.18554842370824]
  In cross-lingual language models, representations for many different languages live in the same space. We compute a task-based measure of cross-lingual alignment in the form of bitext retrieval performance. We examine a range of linguistic, quasi-linguistic, and training-related features as potential predictors of these alignment metrics.
  arXiv  Detail & Related papers  (2021-09-13T21:05:37Z)
• An analysis of full-size Russian complexly NER labelled corpus of Internet user reviews on the drugs based on deep learning and language neural nets [94.37521840642141]
  We present the full-size Russian complexly NER-labeled corpus of Internet user reviews. A set of advanced deep learning neural networks is used to extract pharmacologically meaningful entities from Russian texts.
  arXiv  Detail & Related papers  (2021-04-30T19:46:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.