ForMaT: Dataset for Visually-Grounded Multilingual PDF Translation

• URL: http://arxiv.org/abs/2605.15794v1
• Date: Fri, 15 May 2026 09:50:37 GMT
• Title: ForMaT: Dataset for Visually-Grounded Multilingual PDF Translation
• Abstract summary: We present ForMaT (Format-Preserving Multilingual Translation), a parallel corpus of 3,956 PDFs across 15 language pairs.<n>We employ K-Medoids sampling over 45 geometric features, capturing complex elements like nested tables and formulas to focus only on visually diverse PDF documents.
• Score: 0.27998963147546146
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: We present ForMaT (Format-Preserving Multilingual Translation), a parallel corpus of 3,956 PDFs across 15 language pairs that preserves original layout metadata proposed for multimodal machine translation. To ensure structural diversity in the dataset, we employ K-Medoids sampling over 45 geometric features, capturing complex elements like nested tables and formulas to focus only on visually diverse PDF documents. Our evaluation reveals that current MT systems struggle with spatial grounding and geometric synchronization, often losing the link between text and its visual context. ForMaT provides a benchmark for developing layout-aware translation models that integrate visual and textual context for high-fidelity document reconstruction.

Related papers

• BabelDOC: Better Layout-Preserving PDF Translation via Intermediate Representation [9.979324579114392]
  Existing document translation pipelines face a tension between linguistic processing and layout preservation.<n>We introduce BabelDOC, an Intermediate Representation (IR)-based framework for layout-preserving PDF translation.<n>BabelDOC decouples visual layout metadata from semantic content, enabling document-level translation operations.
  arXiv  Detail & Related papers  (2026-05-11T16:56:44Z)
• Benchmarking Vision-Language Models for French PDF-to-Markdown Conversion [0.14680035572775532]
  This report evaluates PDF-to-Markdown conversion using recent Vision-Language Models (VLMs) on challenging French documents.<n>We introduce a French-focused benchmark of difficult pages selected via model-disagreement sampling from a corpus of 60,000 documents.<n>We observe substantially higher robustness for the strongest proprietary models on handwriting and forms, while several open-weights systems remain competitive on standard printed layouts.
  arXiv  Detail & Related papers  (2026-02-12T13:55:43Z)
• M3DR: Towards Universal Multilingual Multimodal Document Retrieval [0.0]
  We present M3DR (Multilingual Multimodal Document Retrieval), a framework designed to bridge this gap across languages.<n>We generalize across different vision-language architectures and model sizes, enabling robust cross-lingual and cross-modal alignment.<n>Our models, NetraEmbed and ColNetraEmbed achieve state-of-the-art performance with 150% relative improvements on cross-lingual retrieval.
  arXiv  Detail & Related papers  (2025-12-03T07:17:59Z)
• MonkeyOCR v1.5 Technical Report: Unlocking Robust Document Parsing for Complex Patterns [80.05126590825121]
  MonkeyOCR v1.5 is a unified vision-language framework that enhances both layout understanding and content recognition.<n>To address complex table structures, we propose a visual consistency-based reinforcement learning scheme.<n>Two specialized modules, Image-Decoupled Table Parsing and Type-Guided Table Merging, are introduced to enable reliable parsing of tables.
  arXiv  Detail & Related papers  (2025-11-13T15:12:17Z)
• VisR-Bench: An Empirical Study on Visual Retrieval-Augmented Generation for Multilingual Long Document Understanding [49.07705729597171]
  VisR-Bench is a benchmark for question-driven multimodal retrieval in long documents.<n>Our benchmark comprises over 35K high-quality QA pairs across 1.2K documents.<n>We evaluate various retrieval models, including text-based methods, multimodal encoders, and MLLMs.
  arXiv  Detail & Related papers  (2025-08-10T21:44:43Z)
• Towards Visual Text Grounding of Multimodal Large Language Model [74.22413337117617]
  We introduce TRIG, a novel task with a newly designed instruction dataset for benchmarking text-rich image grounding.<n>Specifically, we propose an OCR-LLM-human interaction pipeline to create 800 manually annotated question-answer pairs as a benchmark.<n>A comprehensive evaluation of various MLLMs on our proposed benchmark exposes substantial limitations in their grounding capability on text-rich images.
  arXiv  Detail & Related papers  (2025-04-07T12:01:59Z)
• PDF-WuKong: A Large Multimodal Model for Efficient Long PDF Reading with End-to-End Sparse Sampling [63.93112754821312]
  Multimodal document understanding is a challenging task to process and comprehend large amounts of textual and visual information.<n>Recent advances in Large Language Models (LLMs) have significantly improved the performance of this task.<n>We introduce PDF-WuKong, a multimodal large language model (MLLM) which is designed to enhance multimodal question-answering (QA) for long PDF documents.
  arXiv  Detail & Related papers  (2024-10-08T12:17:42Z)
• M3T: A New Benchmark Dataset for Multi-Modal Document-Level Machine Translation [32.39711525205274]
  Document translation poses a challenge for Neural Machine Translation (NMT) systems. Most document-level NMT systems rely on meticulously curated sentence-level parallel data. Real-world documents often possess intricate text layouts that defy these assumptions. This paper introduces M3T, a novel benchmark dataset tailored to evaluate NMT systems on the comprehensive task of translating semi-structured documents.
  arXiv  Detail & Related papers  (2024-06-12T14:28:25Z)
• DocLLM: A layout-aware generative language model for multimodal document understanding [12.093889265216205]
  We present DocLLM, a lightweight extension to traditional large language models (LLMs) for reasoning over visual documents. Our model focuses exclusively on bounding box information to incorporate the spatial layout structure. We demonstrate that our solution outperforms SotA LLMs on 14 out of 16 datasets across all tasks, and generalizes well to 4 out of 5 previously unseen datasets.
  arXiv  Detail & Related papers  (2023-12-31T22:37:52Z)
• Beyond Triplet: Leveraging the Most Data for Multimodal Machine Translation [53.342921374639346]
  Multimodal machine translation aims to improve translation quality by incorporating information from other modalities, such as vision. Previous MMT systems mainly focus on better access and use of visual information and tend to validate their methods on image-related datasets. This paper establishes new methods and new datasets for MMT.
  arXiv  Detail & Related papers  (2022-12-20T15:02:38Z)
• A High-Quality Multilingual Dataset for Structured Documentation Translation [101.41835967142521]
  This paper presents a high-quality multilingual dataset for the documentation domain. We collect XML-structured parallel text segments from the online documentation for an enterprise software platform.
  arXiv  Detail & Related papers  (2020-06-24T02:08:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.