Benchmarking Vision-Language Models for French PDF-to-Markdown Conversion

• URL: http://arxiv.org/abs/2602.11960v1
• Date: Thu, 12 Feb 2026 13:55:43 GMT
• Title: Benchmarking Vision-Language Models for French PDF-to-Markdown Conversion
• Abstract summary: This report evaluates PDF-to-Markdown conversion using recent Vision-Language Models (VLMs) on challenging French documents.<n>We introduce a French-focused benchmark of difficult pages selected via model-disagreement sampling from a corpus of 60,000 documents.<n>We observe substantially higher robustness for the strongest proprietary models on handwriting and forms, while several open-weights systems remain competitive on standard printed layouts.
• Score: 0.14680035572775532
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This report evaluates PDF-to-Markdown conversion using recent Vision-Language Models (VLMs) on challenging French documents. Document parsing is a critical step for Retrieval-Augmented Generation (RAG) pipelines, where transcription and layout errors propagate to downstream retrieval and grounding. Existing benchmarks often emphasize English or Chinese and can over-penalize benign formatting and linearization choices (e.g., line breaks, list segmentation, alternative table renderings) that are largely irrelevant for downstream use. We introduce a French-focused benchmark of difficult pages selected via model-disagreement sampling from a corpus of 60{,}000 documents, covering handwritten forms, complex layouts, dense tables, and graphics-rich pages. Evaluation is performed with unit-test-style checks that target concrete failure modes (text presence, reading order, and local table constraints) combined with category-specific normalization designed to discount presentation-only variance. Across 15 models, we observe substantially higher robustness for the strongest proprietary models on handwriting and forms, while several open-weights systems remain competitive on standard printed layouts.

Related papers

• MM-BizRAG: Rethinking Multimodal Retrieval-Augmented Generation for General Purpose Enterprise Q&A [13.040489131301081]
  MM-BizRAG proactively extracts and represents document structure via a document structure-aware split.<n> MM-BizRAG consistently outperforms state-of-the-art vision-centric baselines by up to 32% points.
  arXiv  Detail & Related papers  (2026-06-02T21:31:47Z)
• MPDocBench-Parse: Benchmarking Practical Multi-page Document Parsing [74.84107522458798]
  MPDocBench-Parse is a benchmark for multi-page document parsing in real-world applications.<n>It contains 433 manually annotated documents with 3,246 pages, covering 15 document types in English and Chinese, with diverse layout styles.
  arXiv  Detail & Related papers  (2026-05-21T07:36:41Z)
• ForMaT: Dataset for Visually-Grounded Multilingual PDF Translation [0.27998963147546146]
  We present ForMaT (Format-Preserving Multilingual Translation), a parallel corpus of 3,956 PDFs across 15 language pairs.<n>We employ K-Medoids sampling over 45 geometric features, capturing complex elements like nested tables and formulas to focus only on visually diverse PDF documents.
  arXiv  Detail & Related papers  (2026-05-15T09:50:37Z)
• BabelDOC: Better Layout-Preserving PDF Translation via Intermediate Representation [9.979324579114392]
  Existing document translation pipelines face a tension between linguistic processing and layout preservation.<n>We introduce BabelDOC, an Intermediate Representation (IR)-based framework for layout-preserving PDF translation.<n>BabelDOC decouples visual layout metadata from semantic content, enabling document-level translation operations.
  arXiv  Detail & Related papers  (2026-05-11T16:56:44Z)
• HighlightBench: Benchmarking Markup-Driven Table Reasoning in Scientific Documents [88.16322879912458]
  multimodal large language models (MLLMs) have made substantial progress in document understanding, but their ability to treat such cues as explicit logical directives remains under-explored.<n>We introduce HighlightBench, a diagnostic benchmark for markup-driven table understanding that decomposes evaluation into five task families.<n>We show that even strong models remain unstable when visual cues must be consistently aligned with symbolic reasoning under structured output constraints.
  arXiv  Detail & Related papers  (2026-03-25T06:15:40Z)
• FMBench: Adaptive Large Language Model Output Formatting [49.52930069696333]
  We present FMBench, a benchmark for adaptive Markdown output formatting.<n>Experiments on two model families show that SFT consistently improves semantic alignment.<n>Results also reveal an inherent trade-off between semantic and structural objectives.
  arXiv  Detail & Related papers  (2026-02-06T04:42:06Z)
• MonkeyOCR v1.5 Technical Report: Unlocking Robust Document Parsing for Complex Patterns [80.05126590825121]
  MonkeyOCR v1.5 is a unified vision-language framework that enhances both layout understanding and content recognition.<n>To address complex table structures, we propose a visual consistency-based reinforcement learning scheme.<n>Two specialized modules, Image-Decoupled Table Parsing and Type-Guided Table Merging, are introduced to enable reliable parsing of tables.
  arXiv  Detail & Related papers  (2025-11-13T15:12:17Z)
• Logics-Parsing Technical Report [8.982345117231661]
  We propose Logics-Parsing: an end-to-end LVLM-based model augmented with reinforcement learning.<n>Our model incorporates meticulously designed reward mechanisms to optimize complex layout analysis and reading order inference.<n>We introduce LogicsParsingBench, a curated set of 1,078 page-level PDF images spanning nine major categories and over twenty sub-categories.
  arXiv  Detail & Related papers  (2025-09-24T04:54:37Z)
• Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsing [46.14775667559124]
  layoutRL is an end-to-end reinforcement learning framework that trains models to be explicitly layout-aware.<n>We will publicly release our code and dataset to accelerate progress in robust document understanding.
  arXiv  Detail & Related papers  (2025-06-01T15:19:52Z)
• MIRACL-VISION: A Large, multilingual, visual document retrieval benchmark [1.8448587047759064]
  We introduce MIRACL-VISION, a multilingual visual document retrieval evaluation benchmark.<n> MIRACL-VISION covers 18 languages, and is an extension of the MIRACL dataset.<n>We observe a gap in state-of-the-art VLM-based embedding models on multilingual capabilities.
  arXiv  Detail & Related papers  (2025-05-16T19:22:19Z)
• Benchmarking Large Language Models for Handwritten Text Recognition [0.061446808540639365]
  Multimodal Large Language Models (MLLMs) offer a general approach to recognizing diverse handwriting styles without the need for model-specific training.<n>The study benchmarks various proprietary and open-source LLMs against Transkribus models, evaluating their performance on both modern and historical datasets written in English, French, German, and Italian.
  arXiv  Detail & Related papers  (2025-03-19T13:33:29Z)
• Discourse Features Enhance Detection of Document-Level Machine-Generated Content [53.41994768824785]
  Machine-generated content poses challenges such as academic plagiarism and the spread of misinformation.<n>Existing MGC detectors often focus solely on surface-level information, overlooking implicit and structural features.<n>We introduce novel methodologies and datasets to overcome these challenges.
  arXiv  Detail & Related papers  (2024-12-17T08:47:41Z)
• Retrieval is Accurate Generation [99.24267226311157]
  We introduce a novel method that selects context-aware phrases from a collection of supporting documents. Our model achieves the best performance and the lowest latency among several retrieval-augmented baselines.
  arXiv  Detail & Related papers  (2024-02-27T14:16:19Z)
• Are Layout-Infused Language Models Robust to Layout Distribution Shifts? A Case Study with Scientific Documents [54.744701806413204]
  Recent work has shown that infusing layout features into language models (LMs) improves processing of visually-rich documents such as scientific papers. We test whether layout-infused LMs are robust to layout distribution shifts.
  arXiv  Detail & Related papers  (2023-06-01T18:01:33Z)
• Long Document Summarization with Top-down and Bottom-up Inference [113.29319668246407]
  We propose a principled inference framework to improve summarization models on two aspects. Our framework assumes a hierarchical latent structure of a document where the top-level captures the long range dependency. We demonstrate the effectiveness of the proposed framework on a diverse set of summarization datasets.
  arXiv  Detail & Related papers  (2022-03-15T01:24:51Z)
• Towards Making the Most of Context in Neural Machine Translation [112.9845226123306]
  We argue that previous research did not make a clear use of the global context. We propose a new document-level NMT framework that deliberately models the local context of each sentence.
  arXiv  Detail & Related papers  (2020-02-19T03:30:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.