JaWildText: A Benchmark for Vision-Language Models on Japanese Scene Text Understanding

• URL: http://arxiv.org/abs/2603.27942v2
• Date: Tue, 31 Mar 2026 07:50:48 GMT
• Title: JaWildText: A Benchmark for Vision-Language Models on Japanese Scene Text Understanding
• Abstract summary: JaWildText is a diagnostic benchmark for evaluating vision-language models (VLMs) on Japanese scene text understanding.<n>It contains 3,241 instances from 2,961 images newly captured in Japan, with 1.12 million annotated characters spanning 3,643 unique character types.<n>It comprises three complementary tasks that vary in visual organization, output format, and writing style.
• Score: 19.312383956707396
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Japanese scene text poses challenges that multilingual benchmarks often fail to capture, including mixed scripts, frequent vertical writing, and a character inventory far larger than the Latin alphabet. Although Japanese is included in several multilingual benchmarks, these resources do not adequately capture the language-specific complexities. Meanwhile, existing Japanese visual text datasets have primarily focused on scanned documents, leaving in-the-wild scene text underexplored. To fill this gap, we introduce JaWildText, a diagnostic benchmark for evaluating vision-language models (VLMs) on Japanese scene text understanding. JaWildText contains 3,241 instances from 2,961 images newly captured in Japan, with 1.12 million annotated characters spanning 3,643 unique character types. It comprises three complementary tasks that vary in visual organization, output format, and writing style: (i) Dense Scene Text Visual Question Answering (STVQA), which requires reasoning over multiple pieces of visual text evidence; (ii) Receipt Key Information Extraction (KIE), which tests layout-aware structured extraction from mobile-captured receipts; and (iii) Handwriting OCR, which evaluates page-level transcription across various media and writing directions. We evaluate 14 open-weight VLMs and find that the best model achieves an average score of 0.64 across the three tasks. Error analyses show recognition remains the dominant bottleneck, especially for kanji. JaWildText enables fine-grained, script-aware diagnosis of Japanese scene text capabilities, and will be released with evaluation code.

Related papers

• MonkeyOCRv2: A Visual-Text Foundation Model for Document AI [71.06391669976786]
  MonkeyOCRv2 is a visual-text pretrained model for document AI.<n> MonkeyDoc v2 is the largest document-image pretraining corpus, comprising 113 million images spanning 17 languages.
  arXiv  Detail & Related papers  (2026-07-13T13:43:39Z)
• MDPBench: A Benchmark for Multilingual Document Parsing in Real-World Scenarios [72.8160644291677]
  We introduce Multilingual Document Parsing Benchmark, the first benchmark for multilingual digital and photographed document parsing.<n> MDPBench comprises 3,400 document images spanning 17 languages, diverse scripts, and varied photographic conditions.
  arXiv  Detail & Related papers  (2026-03-30T07:47:46Z)
• SEA-Vision: A Multilingual Benchmark for Comprehensive Document and Scene Text Understanding in Southeast Asia [40.4434142867308]
  We introduce SEA-Vision, a benchmark that jointly evaluates Document Parsing and Text-Centric Visual Question Answering (TEC-VQA) across 11 Southeast Asian languages.<n>SEA-Vision contains 15,234 document parsing pages from nine representative document types, annotated with hierarchical page-, block-, and line-level labels.<n>It also provides 7,496 TEC-VQA question-answer pairs that probe text recognition, numerical calculation, comparative analysis, logical reasoning, and spatial understanding.
  arXiv  Detail & Related papers  (2026-03-16T15:21:12Z)
• Bharat Scene Text: A Novel Comprehensive Dataset and Benchmark for Indian Language Scene Text Understanding [3.40492636281753]
  Bharat Scene Text dataset is a comprehensive benchmark for studying Indian Language Scene Text Recognition.<n>It comprises more than 100K words that span 11 Indian languages and English, sourced from over 6,500 scene images captured across various linguistic regions of India.<n>The dataset is meticulously annotated and supports multiple scene text tasks, including Scene Text Detection, Script Identification, Cropped Word Recognition, and End-to-End Scene Text Recognition.
  arXiv  Detail & Related papers  (2025-11-28T10:58:37Z)
• EasyText: Controllable Diffusion Transformer for Multilingual Text Rendering [9.087419148444225]
  This paper introduces EasyText, a text rendering framework based on DiT (Diffusion Transformer)<n>We propose character positioning encoding and position encoding techniques to achieve controllable and precise text rendering.<n>We construct a large-scale synthetic text image dataset with 1 million multilingual image-text annotations as well as a high-quality dataset of 20K annotated images.
  arXiv  Detail & Related papers  (2025-05-30T09:55:39Z)
• KhmerST: A Low-Resource Khmer Scene Text Detection and Recognition Benchmark [1.5409800688911346]
  We introduce the first Khmer scene-text dataset, featuring 1,544 expert-annotated images. This diverse dataset includes flat text, raised text, poorly illuminated text, distant polygon and partially obscured text.
  arXiv  Detail & Related papers  (2024-10-23T21:04:24Z)
• Dataset and Benchmark for Urdu Natural Scenes Text Detection, Recognition and Visual Question Answering [50.52792174648067]
  This initiative seeks to bridge the gap between textual and visual comprehension. We propose a new multi-task Urdu scene text dataset comprising over 1000 natural scene images. We provide fine-grained annotations for text instances, addressing the limitations of previous datasets.
  arXiv  Detail & Related papers  (2024-05-21T06:48:26Z)
• MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answering [57.30218240464696]
  We introduce MTVQA, the first benchmark featuring high-quality human expert annotations across 9 diverse languages.<n>MTVQA is the first benchmark featuring high-quality human expert annotations across 9 diverse languages.
  arXiv  Detail & Related papers  (2024-05-20T12:35:01Z)
• The First Swahili Language Scene Text Detection and Recognition Dataset [55.83178123785643]
  There is a significant gap in low-resource languages, especially the Swahili Language. Swahili is widely spoken in East African countries but is still an under-explored language in scene text recognition. We propose a comprehensive dataset of Swahili scene text images and evaluate the dataset on different scene text detection and recognition models.
  arXiv  Detail & Related papers  (2024-05-19T03:55:02Z)
• A Benchmark for Chinese-English Scene Text Image Super-resolution [15.042152725255171]
  Scene Text Image Super-resolution (STISR) aims to recover high-resolution (HR) scene text images with visually pleasant and readable text content from low-resolution (LR) input. Most existing works focus on recovering English texts, which have relatively simple character structures. We propose a real-world Chinese-English benchmark dataset, namely Real-CE, for the task of STISR.
  arXiv  Detail & Related papers  (2023-08-07T02:57:48Z)
• AE TextSpotter: Learning Visual and Linguistic Representation for Ambiguous Text Spotting [98.08853679310603]
  This work proposes a novel text spotter, named Ambiguity Eliminating Text Spotter (AE TextSpotter) AE TextSpotter learns both visual and linguistic features to significantly reduce ambiguity in text detection. To our knowledge, it is the first time to improve text detection by using a language model.
  arXiv  Detail & Related papers  (2020-08-03T08:40:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.