TajPersLexon: A Tajik-Persian Lexical Resource and Hybrid Model for Cross-Script Low-Resource NLP

• URL: http://arxiv.org/abs/2605.06886v1
• Date: Thu, 07 May 2026 19:37:34 GMT
• Title: TajPersLexon: A Tajik-Persian Lexical Resource and Hybrid Model for Cross-Script Low-Resource NLP
• Abstract summary: This work introduces TajPersLexon, a curated Tajik-Persian parallel lexical resource of 40,112 word and short-phrase pairs.<n>We compare three methodological families: (i) a lightweight hybrid pipeline, (ii) neural sequence-to-sequence models, and (iii) retrieval methods.<n>Our evaluation establishes that the task is essentially solvable, with neural and retrieval baselines achieving 98-99% top-1 accuracy.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This work introduces TajPersLexon, a curated Tajik--Persian parallel lexical resource of 40,112 word and short-phrase pairs for cross-script lexical retrieval, transliteration, and alignment in low-resource settings. We conduct a comprehensive CPU-only benchmark comparing three methodological families: (i) a lightweight hybrid pipeline, (ii) neural sequence-to-sequence models, and (iii) retrieval methods. Our evaluation establishes that the task is essentially solvable, with neural and retrieval baselines achieving 98-99% top-1 accuracy. Crucially, we demonstrate that while large multilingual sentence transformers fail on this exact lexical matching, our interpretable hybrid model offers a favorable accuracy-efficiency trade-off for practical applications, achieving 96.4% accuracy in an OCR post-correction task. All experiments use fixed random seeds for full reproducibility. The dataset, code, and models will be publicly released.

Related papers

• Character-Level Transformer for Tajik-Persian Transliteration with a Parallel Lexical Corpus [0.0]
  This study addresses automatic transliteration from Tajik (Cyrillic script) to Persian (Perso-Arabic script)<n>We present a curated, lexicographically verified parallel corpus of 52,152 Tajik--Persian words and short phrases.
  arXiv  Detail & Related papers  (2026-05-09T17:57:29Z)
• A Hybrid Protocol for Large-Scale Semantic Dataset Generation in Low-Resource Languages: The Turkish Semantic Relations Corpus [0.0]
  We present a hybrid methodology for generating large-scale semantic relationship datasets in low-resource languages.<n>Our approach integrates FastText embeddings with Agglomerative Clustering to identify semantic clusters, Gemini 2.5-Flash for automated semantic relationship classification, and integration with curated dictionary sources.<n>The resulting dataset comprises 843,000 unique Turkish semantic pairs representing a 10x scale increase over existing resources at minimal cost.
  arXiv  Detail & Related papers  (2026-01-19T17:38:52Z)
• NagaNLP: Bootstrapping NLP for Low-Resource Nagamese Creole with Human-in-the-Loop Synthetic Data [6.689013010749215]
  NagaNLP is a comprehensive open-source toolkit for Nagamese.<n>It relies on LLM-driven but human-validated synthetic data generation.<n>We train both discriminative and generative models.
  arXiv  Detail & Related papers  (2025-12-14T04:08:26Z)
• A Theoretically Grounded Hybrid Ensemble for Reliable Detection of LLM-Generated Text [0.0]
  We propose a theoretically grounded hybrid ensemble that fuses three complementary detection paradigms.<n>The core novelty lies in an optimized weighted voting framework, where ensemble weights are learned on the probability simplex to maximize F1-score.<n>Our system achieves 94.2% accuracy and an AUC of 0.978, with a 35% relative reduction in false positives on academic text.
  arXiv  Detail & Related papers  (2025-11-27T06:42:56Z)
• Reproducible Machine Learning-based Voice Pathology Detection: Introducing the Pitch Difference Feature [1.7779568951268254]
  We introduce a novel methodology for voice pathology detection using the publicly available Saarbr"ucken Voice Database.<n>We evaluate six machine learning (ML) algorithms -- support vector machine, k-nearest neighbors, naive Bayes, decision tree, random forest, and AdaBoost.<n>Our approach 85.61%, 84.69% and 85.22% unweighted average recall (UAR) for females, males and combined results respectively.
  arXiv  Detail & Related papers  (2024-10-14T14:17:52Z)
• Language Rectified Flow: Advancing Diffusion Language Generation with Probabilistic Flows [53.31856123113228]
  This paper proposes Language Rectified Flow (ours) Our method is based on the reformulation of the standard probabilistic flow models. Experiments and ablation studies demonstrate that our method can be general, effective, and beneficial for many NLP tasks.
  arXiv  Detail & Related papers  (2024-03-25T17:58:22Z)
• TriSum: Learning Summarization Ability from Large Language Models with Structured Rationale [66.01943465390548]
  We introduce TriSum, a framework for distilling large language models' text summarization abilities into a compact, local model. Our method enhances local model performance on various benchmarks. It also improves interpretability by providing insights into the summarization rationale.
  arXiv  Detail & Related papers  (2024-03-15T14:36:38Z)
• Ensemble Transfer Learning for Multilingual Coreference Resolution [60.409789753164944]
  A problem that frequently occurs when working with a non-English language is the scarcity of annotated training data. We design a simple but effective ensemble-based framework that combines various transfer learning techniques. We also propose a low-cost TL method that bootstraps coreference resolution models by utilizing Wikipedia anchor texts.
  arXiv  Detail & Related papers  (2023-01-22T18:22:55Z)
• OneAligner: Zero-shot Cross-lingual Transfer with One Rich-Resource Language Pair for Low-Resource Sentence Retrieval [91.76575626229824]
  We present OneAligner, an alignment model specially designed for sentence retrieval tasks. When trained with all language pairs of a large-scale parallel multilingual corpus (OPUS-100), this model achieves the state-of-the-art result. We conclude through empirical results and analyses that the performance of the sentence alignment task depends mostly on the monolingual and parallel data size.
  arXiv  Detail & Related papers  (2022-05-17T19:52:42Z)
• Task-agnostic Continual Learning with Hybrid Probabilistic Models [75.01205414507243]
  We propose HCL, a Hybrid generative-discriminative approach to Continual Learning for classification. The flow is used to learn the data distribution, perform classification, identify task changes, and avoid forgetting. We demonstrate the strong performance of HCL on a range of continual learning benchmarks such as split-MNIST, split-CIFAR, and SVHN-MNIST.
  arXiv  Detail & Related papers  (2021-06-24T05:19:26Z)
• Tackling the Low-resource Challenge for Canonical Segmentation [23.17111619633273]
  Canonical morphological segmentation consists of dividing words into their standardized morphemes. We explore two new models for the task, borrowing from the closely related area of morphological generation. We find that, in the low-resource setting, the novel approaches outperform existing ones on all languages by up to 11.4% accuracy.
  arXiv  Detail & Related papers  (2020-10-06T15:15:05Z)
• Parameter Space Factorization for Zero-Shot Learning across Tasks and Languages [112.65994041398481]
  We propose a Bayesian generative model for the space of neural parameters. We infer the posteriors over such latent variables based on data from seen task-language combinations. Our model yields comparable or better results than state-of-the-art, zero-shot cross-lingual transfer methods.
  arXiv  Detail & Related papers  (2020-01-30T16:58:56Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.