Evaluation of Chunking Strategies for Effective Text Embedding in Low-Resource Language on Agricultural Documents

• URL: http://arxiv.org/abs/2605.22203v1
• Date: Thu, 21 May 2026 09:06:13 GMT
• Title: Evaluation of Chunking Strategies for Effective Text Embedding in Low-Resource Language on Agricultural Documents
• Abstract summary: Performance is evaluated using four metrics: Average Retrieval Score (L2 distance), Answer Relevance, Khmer Coverage, and Khmer Intersection over Union.<n>We observe the best performance for the character-based Recursive chunking method with a chunk size of 300 characters.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In this study, we compare the performance of four text chunking approaches: Recursive, Khmer-Aware, Sentence-Based, and LLM-Based within a Retrieval-Augmented Generation (RAG) framework applied to Khmer agricultural documents. The document chunks are encoded using the BGE-M3 multilingual embedding model and retrieved using the FAISS library. Performance is evaluated using four metrics: Average Retrieval Score (L2 distance), Answer Relevance, Khmer Coverage, and Khmer Intersection over Union, all measured against ground-truth question-answer pairs. For evaluation, we perform 5-fold cross-validation over 18 question-answer pairs. We observe the best performance for the character-based Recursive chunking method with a chunk size of 300 characters, achieving the lowest L2 distance (0.4295 +- 0.0461), highest Answer Relevance (0.8663 +- 0.0199), and highest Khmer IoU (0.6441 +- 0.0347). A paired t-test shows a statistically significant improvement over the Sentence-Based chunking method in L2 distance (p = 0.0121). These results highlight the importance of segmentation granularity and structural preservation for optimizing dense retrieval in morphologically complex, low-resource languages such as Khmer.

Related papers

• Query Translation vs. Cross-Lingual Embeddings for Sinhala-Tamil E-Government Information Retrieval [0.0]
  This paper presents a comparative evaluation of cross-lingual information retrieval methods.<n>Two CLIR paradigms are investigated: Query Translation (QT), employing Google Translate, NLLB, and mBART50, and Cross-Lingual Embeddings (CLE)<n>BGE-M3 achieves the highest Recall@15, reaching 96.2% for Sinhala-English and 95.6% for Tamil-English.
  arXiv  Detail & Related papers  (2026-08-13T04:49:39Z)
• A Comparative Study of Language Models for Khmer Retrieval-Augmented Question Answering [0.0]
  Retrieval-Augmented Generation (RAG) has emerged as a promising paradigm for grounding large language model (LLM) outputs in retrieved evidence.<n>We present a RAG-based question answering system for Khmer-language telecom-domain documents.
  arXiv  Detail & Related papers  (2026-05-21T07:36:01Z)
• SlovKE: A Large-Scale Dataset and LLM Evaluation for Slovak Keyphrase Extraction [0.00954904463032233]
  Keyphrase extraction for morphologically rich, low-resource languages remains understudied.<n>We construct a dataset of 227,432 scientific abstracts with author-assigned keyphrases.<n>KeyLLM produces keyphrases closer to the canonical forms assigned by authors.<n>Our analysis identifies morphological mismatch as the dominant failure mode for statistical methods.
  arXiv  Detail & Related papers  (2026-03-16T16:47:45Z)
• Towards Global Retrieval Augmented Generation: A Benchmark for Corpus-Level Reasoning [50.27838512822097]
  We introduce GlobalQA, the first benchmark specifically designed to evaluate global RAG capabilities.<n>We propose GlobalRAG, a multi-tool collaborative framework that preserves structural coherence through chunk-level retrieval.<n>On the Qwen2.5-14B model, GlobalRAG achieves 6.63 F1 compared to the strongest baseline's 1.51 F1.
  arXiv  Detail & Related papers  (2025-10-30T07:29:14Z)
• TextMine: Data, Evaluation Framework and Ontology-guided LLM Pipeline for Humanitarian Mine Action [4.990484801014005]
  Humanitarian Mine Action (HMA) addresses the challenge of detecting and removing landmines from conflict regions.<n>Much of the life-saving operational knowledge produced by HMA agencies is buried in unstructured reports.<n>To address this issue, we propose TextMine: the first dataset, evaluation framework and ontology-guided large language model (LLM) pipeline.
  arXiv  Detail & Related papers  (2025-09-18T15:55:19Z)
• Vuyko Mistral: Adapting LLMs for Low-Resource Dialectal Translation [0.0]
  This paper introduces the first effort to adapt large language models to the Ukrainian dialect Hutsul.<n>We created a parallel corpus of 9852 dialect-to-standard Ukrainian sentence pairs and a dictionary of 7320 dialectal word mappings.
  arXiv  Detail & Related papers  (2025-06-09T10:30:35Z)
• MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigm [60.14048367611333]
  MonkeyOCR is a vision-language model for document parsing.<n>It advances the state of the art by leveraging a Structure-Recognition-Relation (SRR) triplet paradigm.
  arXiv  Detail & Related papers  (2025-06-05T16:34:57Z)
• Optimized Text Embedding Models and Benchmarks for Amharic Passage Retrieval [49.1574468325115]
  We introduce Amharic-specific dense retrieval models based on pre-trained Amharic BERT and RoBERTa backbones.<n>Our proposed RoBERTa-Base-Amharic-Embed model (110M parameters) achieves a 17.6% relative improvement in MRR@10.<n>More compact variants, such as RoBERTa-Medium-Amharic-Embed (42M) remain competitive while being over 13x smaller.
  arXiv  Detail & Related papers  (2025-05-25T23:06:20Z)
• Enhancing Cross-Language Code Translation via Task-Specific Embedding Alignment in Retrieval-Augmented Generation [1.64043572114825]
  We introduce a novel method to enhance cross-language code translation from Fortran to C++ by integrating task-specific embedding alignment.<n>Our strategy aligns the retrieval model directly with the objective of maximizing translation quality, as quantified by the CodeBLEU metric.<n>By integrating these CodeBLEU-optimized embeddings into the RAG framework, our approach significantly enhances both retrieval accuracy and code generation quality.
  arXiv  Detail & Related papers  (2024-12-06T16:22:32Z)
• BooookScore: A systematic exploration of book-length summarization in the era of LLMs [53.42917858142565]
  We develop an automatic metric, BooookScore, that measures the proportion of sentences in a summary that do not contain any of the identified error types. We find that closed-source LLMs such as GPT-4 and 2 produce summaries with higher BooookScore than those generated by open-source models.
  arXiv  Detail & Related papers  (2023-10-01T20:46:44Z)
• Cross-lingual Cross-temporal Summarization: Dataset, Models, Evaluation [20.675242617417677]
  Cross-lingual cross-temporal summarization (CLCTS) is a largely unexplored area that has the potential to improve cross-cultural accessibility and understanding. This paper comprehensively addresses the CLCTS task, including dataset creation, modeling, and evaluation.
  arXiv  Detail & Related papers  (2023-06-22T14:31:18Z)
• SEAHORSE: A Multilingual, Multifaceted Dataset for Summarization Evaluation [52.186343500576214]
  We introduce SEAHORSE, a dataset for multilingual, multifaceted summarization evaluation. SEAHORSE consists of 96K summaries with human ratings along 6 dimensions of text quality. We show that metrics trained with SEAHORSE achieve strong performance on the out-of-domain meta-evaluation benchmarks TRUE and mFACE.
  arXiv  Detail & Related papers  (2023-05-22T16:25:07Z)
• MOCHA: A Dataset for Training and Evaluating Generative Reading Comprehension Metrics [55.85042753772513]
  We introduce a benchmark for training and evaluating generative reading comprehension metrics: MOdeling Correctness with Human. s. Using MOCHA, we train a Learned Evaluation metric for Reading Pearson, LERC, to mimic human judgement scores. LERC outperforms baseline metrics by 10 to 36 absolute points on held-out annotations. When we evaluate on minimal pairs, LERC achieves 80% accuracy, outperforming baselines by 14 to 26 absolute percentage points while leaving significant room for improvement.
  arXiv  Detail & Related papers  (2020-10-07T20:22:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.