KoVRE: Training an Efficient Embedding Model for Korean Visual Document Retrieval

• URL: http://arxiv.org/abs/2608.01389v1
• Date: Sun, 02 Aug 2026 17:11:09 GMT
• Title: KoVRE: Training an Efficient Embedding Model for Korean Visual Document Retrieval
• Abstract summary: We introduce KoVRE: Korean Visual Document Retrieval Embedding, a single-vector retriever for Korean visual documents.<n>We train the model on 708,729 Korean and English query-page pairs using positive-aware hard-negative mining.<n>Our results demonstrate that targeted bilingual supervision and our carefully designed training strategies can produce a highly effective Korean VDR model.
• Score: 2.7900391222798295
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Visual Document Retrieval (VDR) directly matches text queries against document images, preserving visual and structural information that may be lost during text extraction. However, existing VDR models and training resources remain predominantly English-centric, while many high-performing systems rely on massive backbones or storage-intensive multi-vector representations. To address these limitations, we introduce KoVRE: Korean Visual Document Retrieval Embedding, a single-vector retriever for Korean visual documents, alongside a comprehensive training recipe. We train the model on 708,729 Korean and English query-page pairs using positive-aware hard-negative mining and conduct controlled analyses of training-data composition, hard-negative treatment, and reranker-based knowledge distillation. Across Korean visual document retrieval benchmarks, our 2B model substantially improves over the base backbone model, outperforming both its 8B single-vector counterpart and a strong multi-vector baseline. These results demonstrate that targeted bilingual supervision and our carefully designed training strategies can produce a highly effective Korean VDR model across diverse document domains, without requiring a scaled-up backbone or multi-vector representations.

Related papers

• KoViDoRe: Korean Visual Document Retrieval [3.5302061668374036]
  We introduce KoViDoRe, a benchmark for Korean visual document retrieval.<n>The dataset is constructed from publicly available Korean documents with diverse layouts, including tables, figures, and multi-column structures.<n>We also curate a large-scale training dataset, Ko-VDR Train Public, to support the development of retrieval models tailored to Korean visual documents.
  arXiv  Detail & Related papers  (2026-08-21T08:03:59Z)
• Beyond the Grid: Layout-Informed Multi-Vector Retrieval with Parsed Visual Document Representations [39.98860473310998]
  ColParse is a novel paradigm that leverages a document parsing model to generate a small set of layout-informed sub-image embeddings.<n>Experiments demonstrate that our method reduces storage requirements by over 95% while simultaneously yielding significant performance gains.
  arXiv  Detail & Related papers  (2026-03-02T09:55:00Z)
• Towards Khmer Scene Document Layout Detection [3.5477182055025107]
  We present the first comprehensive study on Khmer scene document layout detection.<n>We contribute a novel framework comprising three key elements: (1) a robust training and benchmarking dataset specifically for Khmer scene layouts; (2) an open-source document augmentation tool capable of synthesizing realistic scene documents to scale training data; and (3) layout detection baselines utilizing YOLO-based architectures with oriented bounding boxes (OBB) to handle geometric distortions.
  arXiv  Detail & Related papers  (2026-02-28T15:30:16Z)
• CausalEmbed: Auto-Regressive Multi-Vector Generation in Latent Space for Visual Document Embedding [71.88471147281406]
  We propose an auto-regressive generation approach, CausalEmbed, for constructing multi-vector embeddings.<n>By incorporating iterative margin loss during contrastive training, CausalEmbed encourages embedding models to learn compact and well-structured representations.<n>Our method enables efficient VDR tasks using only dozens of visual tokens, achieving a 30-155x reduction in token count.
  arXiv  Detail & Related papers  (2026-01-29T04:47:27Z)
• SDS KoPub VDR: A Benchmark Dataset for Visual Document Retrieval in Korean Public Documents [10.146296597660598]
  Existing benchmarks for visual document retrieval (VDR) largely overlook non-English languages and the structural complexity of official publications.<n>We introduce SDS KoPub VDR, the first large-scale, public benchmark for retrieving and understanding Korean public documents.<n>The benchmark is built upon 361 real-world documents, including 256 files under the KOGL Type 1 license and 105 from official legal portals.
  arXiv  Detail & Related papers  (2025-11-07T01:16:07Z)
• Vision-Free Retrieval: Rethinking Multimodal Search with Textual Scene Descriptions [81.33113485830711]
  We introduce a vision-free, single-encoder retrieval pipeline for vision-language models.<n>We migrate to a text-to-text paradigm with the assistance of VLLM-generated structured image descriptions.<n>Our approach achieves state-of-the-art zero-shot performance on multiple retrieval and compositionality benchmarks.
  arXiv  Detail & Related papers  (2025-09-23T16:22:27Z)
• Harnessing PDF Data for Improving Japanese Large Multimodal Models [56.80385809059738]
  Large Multimodal Models (LMMs) have demonstrated strong performance in English, but their effectiveness in Japanese remains limited.<n>Current Japanese LMMs often rely on translated English datasets, restricting their ability to capture Japan-specific cultural knowledge.<n>We introduce a fully automated pipeline that leverages pretrained models to extract image-text pairs from PDFs.
  arXiv  Detail & Related papers  (2025-02-20T17:59:59Z)
• OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models [122.27878464009181]
  We conducted a comprehensive evaluation of Large Multimodal Models, such as GPT4V and Gemini, in various text-related visual tasks. OCRBench contains 29 datasets, making it the most comprehensive OCR evaluation benchmark available.
  arXiv  Detail & Related papers  (2023-05-13T11:28:37Z)
• Reading Chinese in Natural Scenes with a Bag-of-Radicals Prior [0.0]
  We evaluate a series of classic methods performing well on Latin datasets, finding a significant performance drop. To improve the performance on Chinese datasets, we propose a novel radical-embedding (RE) representation. In addition, we utilize a bag of radicals as supervision signals for multi-task training to improve the ideographic structure perception of our model.
  arXiv  Detail & Related papers  (2022-10-05T21:56:09Z)
• Unsupervised Vision-and-Language Pre-training via Retrieval-based Multi-Granular Alignment [66.77841319057299]
  We propose a novel unsupervised Vision-and-Language pre-training curriculum for non-parallel texts and images. We first construct a weakly aligned image-text corpus via a retrieval-based approach, then apply a set of multi-granular alignment pre-training tasks. A comprehensive ablation study shows each granularity is helpful to learn a stronger pre-trained model.
  arXiv  Detail & Related papers  (2022-03-01T05:34:01Z)
• Unsupervised Vision-and-Language Pre-training Without Parallel Images and Captions [92.47566804182338]
  We investigate if a strong V&L representation model can be learned through unsupervised pre-training without image-caption corpora. In particular, we propose to conduct mask-and-predict'' pre-training on text-only and image-only corpora. We find that such a simple approach performance close to a model pre-trained with aligned data, on four English V&L benchmarks.
  arXiv  Detail & Related papers  (2020-10-24T08:17:54Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.