Do MLLMs Really Understand Low-Resource Khmer Documents? A Pilot Study on Khmer Document VQA

• URL: http://arxiv.org/abs/2608.28635v1
• Date: Sat, 08 Aug 2026 15:52:31 GMT
• Title: Do MLLMs Really Understand Low-Resource Khmer Documents? A Pilot Study on Khmer Document VQA
• Abstract summary: This paper presents a pilot diagnostic evaluation of open MLLMs on Khmer document images.<n> Khmer-script answers remain substantially more difficult than English and numeric fields.
• Score: 0.8594140167290097
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Recent multimodal large language models (MLLMs) have advanced document understanding, visual question answering, and text extraction. However, their reliability in low-resource, non-Latin settings remains uncertain. Khmer form documents present particular challenges because they contain complex script forms, mixed Khmer-English fields, and monetary values in both Cambodian Riel and US Dollars. Available resources for Khmer Document VQA are also limited. This paper presents a pilot diagnostic evaluation of open MLLMs on Khmer document images. We construct an evaluation subset from the previously introduced KH-FUNSD collection, covering invoices, receipts, quotations, and other business forms. The subset includes questions in English and Khmer, with answers retained in their original English, Khmer, mixed-script, or numeric forms. Rather than introducing a full public benchmark, this study examines the capabilities and failure modes of existing models. We evaluate representative open Qwen-VL models using direct image-based prompting and compare parser-assisted and external OCR-assisted configurations with Qwen3-VL-8B. Direct Qwen3-VL-8B outperforms smaller models, achieving 51.9% overall accuracy, although performance remains limited for Khmer-script and mixed-script answers. External OCR produces the strongest results, reaching 61.9% with Tesseract and 61.6% with PaddleOCR. Nevertheless, Khmer-script answers remain substantially more difficult than English and numeric fields. The results indicate that current MLLMs can process visually clear English and structured numeric content, but reliable native Khmer document understanding remains an open challenge.

Related papers

• BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models [37.78339878369385]
  We introduce BEAR-Bench, a self-contained, complex English-and-Russian benchmark comprising 1000 human-annotated questions.<n>We evaluate 16 proprietary and open-weight MLLMs, including Gemini 3.1 Pro and Qwen3.5-397B, on BEAR-Bench and observe clear headroom even for the strongest systems.
  arXiv  Detail & Related papers  (2026-08-18T15:29:09Z)
• TongGuOCR: A Layout-Aware and Token-Augmented OCR MLLM for Chinese Historical Documents [50.64307290680222]
  TongGuOCR is a layout-aware and token-augmented multimodal large language model (MLLM) for OCR of Chinese historical documents.<n>TongGuOCR achieves 93.76 AR and reduces NED from 10.43 to 6.15 and RO-ED from 7.53 to 3.49 relative to the best competing score for each metric.
  arXiv  Detail & Related papers  (2026-08-08T04:50:00Z)
• KazakhOCR: A Synthetic Benchmark for Evaluating Multimodal Models in Low-Resource Kazakh Script OCR [0.0]
  Kazakh is a Turkic language using the Arabic, Cyrillic, and Latin scripts, making it unique in terms of optical character recognition (OCR)<n>Work on OCR for low-resource Kazakh scripts is very scarce, and no OCR benchmarks or images exist for the Arabic and Latin scripts.<n>We construct a synthetic OCR dataset of 7,219 images for all three scripts with font, color, and noise variations to imitate real OCR tasks.
  arXiv  Detail & Related papers  (2026-02-17T14:24:49Z)
• MosaicDoc: A Large-Scale Bilingual Benchmark for Visually Rich Document Understanding [7.650139800950797]
  MosaicDoc is a large-scale, bilingual (Chinese and English) resource designed to push the boundaries of Visually Rich Document Understanding (VRDU)<n>With 72K images and over 600K QA pairs, MosaicDoc serves as a definitive benchmark for the field.<n>Our evaluation of state-of-the-art models on this benchmark reveals their current limitations in handling real-world document complexity.
  arXiv  Detail & Related papers  (2025-11-13T03:34:44Z)
• CUS-QA: Local-Knowledge-Oriented Open-Ended Question Answering Dataset [1.4999444543328293]
  We introduce CUS-QA, a benchmark for open-ended regional question answering.<n>We provide strong baselines using state-of-the-art large language models (LLMs)<n>We evaluate state-of-the-art LLMs through prompting and complement this with human judgments of answer correctness.
  arXiv  Detail & Related papers  (2025-07-30T15:10:55Z)
• M-DocSum: Do LVLMs Genuinely Comprehend Interleaved Image-Text in Document Summarization? [49.53982792497275]
  We investigate whether Large Vision-Language Models (LVLMs) genuinely comprehend interleaved image-text in the document.<n>Existing document understanding benchmarks often assess LVLMs using question-answer formats.<n>We introduce a novel and challenging Multimodal Document Summarization Benchmark (M-DocSum-Bench)<n>M-DocSum-Bench comprises 500 high-quality arXiv papers, along with interleaved multimodal summaries aligned with human preferences.
  arXiv  Detail & Related papers  (2025-03-27T07:28:32Z)
• Vision-Language Models Do Not Understand Negation [50.27667000027403]
  NegBench is a benchmark designed to evaluate negation understanding across 18 task variations and $79$k examples.<n>We show that this approach can result in a 10% increase in recall on negated queries and a 28% boost in accuracy on multiple-choice questions with negated captions.
  arXiv  Detail & Related papers  (2025-01-16T09:55:42Z)
• CC-OCR: A Comprehensive and Challenging OCR Benchmark for Evaluating Large Multimodal Models in Literacy [50.78228433498211]
  CC-OCR comprises four OCR-centric tracks: multi-scene text reading, multilingual text reading, document parsing, and key information extraction.<n>It includes 39 subsets with 7,058 full annotated images, of which 41% are sourced from real applications, and released for the first time.<n>We evaluate nine prominent LMMs and reveal both the strengths and weaknesses of these models, particularly in text grounding, multi-orientation, and hallucination of repetition.
  arXiv  Detail & Related papers  (2024-12-03T07:03:25Z)
• INDIC QA BENCHMARK: A Multilingual Benchmark to Evaluate Question Answering capability of LLMs for Indic Languages [25.402797722575805]
  Indic QA Benchmark is a dataset for context grounded question answering in 11 major Indian languages.<n> Evaluations revealed weak performance in low resource languages due to a strong English language bias in their training data.<n>We also investigated the Translate Test paradigm,where inputs are translated to English for processing and the results are translated back into the source language for output.
  arXiv  Detail & Related papers  (2024-07-18T13:57:16Z)
• CaLMQA: Exploring culturally specific long-form question answering across 23 languages [58.18984409715615]
  CaLMQA is a dataset of 51.7K culturally specific questions across 23 different languages.<n>We evaluate factuality, relevance and surface-level quality of LLM-generated long-form answers.
  arXiv  Detail & Related papers  (2024-06-25T17:45:26Z)
• mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding [55.4806974284156]
  Document understanding refers to automatically extract, analyze and comprehend information from digital documents, such as a web page. Existing Multi-model Large Language Models (MLLMs) have demonstrated promising zero-shot capabilities in shallow OCR-free text recognition.
  arXiv  Detail & Related papers  (2023-07-04T11:28:07Z)
• Cross-Lingual Question Answering over Knowledge Base as Reading Comprehension [61.079852289005025]
  Cross-lingual question answering over knowledge base (xKBQA) aims to answer questions in languages different from that of the provided knowledge base. One of the major challenges facing xKBQA is the high cost of data annotation. We propose a novel approach for xKBQA in a reading comprehension paradigm.
  arXiv  Detail & Related papers  (2023-02-26T05:52:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.