DocReward: A Document Reward Model for Structuring and Stylizing

• URL: http://arxiv.org/abs/2510.11391v1
• Date: Mon, 13 Oct 2025 13:36:32 GMT
• Title: DocReward: A Document Reward Model for Structuring and Stylizing
• Authors: Junpeng Liu, Yuzhong Zhao, Bowen Cao, Jiayu Ding, Yilin Jia, Tengchao Lv, Yupan Huang, Shaohan Huang, Nan Yang, Li Dong, Lei Cui, Tao Ge, Xun Wang, Huitian Jiao, Sun Mao, FNU Kartik, Si-Qing Chen, Wai Lam, Furu Wei,
• Abstract summary: DocReward is a document reward model that evaluates documents based on their structure and style.<n>It is trained using the Bradley-Terry loss to score documents, penalizing predictions that contradict the annotated ranking.<n>It achieves a significantly higher win rate of 60.8%, compared to GPT-5's 37.7% win rate.
• Score: 107.03974018371058
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Recent advances in agentic workflows have enabled the automation of tasks such as professional document generation. However, they primarily focus on textual quality, neglecting visual structure and style, which are crucial for readability and engagement. This gap arises mainly from the absence of suitable reward models to guide agentic workflows toward producing documents with stronger structural and stylistic quality. To address this, we propose DocReward, a document reward model that evaluates documents based on their structure and style. We construct a multi-domain dataset DocPair of 117K paired documents, covering 32 domains and 267 document types, each including a high- and low-professionalism document with identical content but different structure and style. This enables the model to evaluate professionalism comprehensively, and in a textual-quality-agnostic way. DocReward is trained using the Bradley-Terry loss to score documents, penalizing predictions that contradict the annotated ranking. To assess the performance of reward models, we create a test dataset containing document bundles ranked by well-educated human evaluators. Notably, DocReward outperforms GPT-4o and GPT-5 in accuracy by 30.6 and 19.4 percentage points, respectively, demonstrating its superiority over baselines. In an extrinsic evaluation of document generation, DocReward achieves a significantly higher win rate of 60.8%, compared to GPT-5's 37.7% win rate, demonstrating its utility in guiding generation agents toward producing human-preferred documents.

Related papers

• On Finding Inconsistencies in Documents [6.773356807601893]
  We introduce a benchmark, FIND (Finding INconsistencies in Documents), where each example is a document with an inconsistency inserted manually by a domain expert.<n>Despite the documents being long, technical, and complex, the best-performing model (gpt-5) recovered 64% of the inserted inconsistencies.
  arXiv  Detail & Related papers  (2025-12-21T05:20:21Z)
• DOCR-Inspector: Fine-Grained and Automated Evaluation of Document Parsing with VLM [35.910677096654574]
  Document parsing aims to transform unstructured PDF images into semi-structured data, facilitating the digitization and utilization of information in diverse domains.<n>Common practice often selects the top-performing model on standard benchmarks.<n>We introduce DOCR-Inspector, which formalizes document parsing assessment as fine-grained error detection and analysis.
  arXiv  Detail & Related papers  (2025-12-11T13:16:33Z)
• Improving Document Retrieval Coherence for Semantically Equivalent Queries [63.97649988164166]
  We propose a variation of the Multi-Negative Ranking loss for training DR that improves the coherence of models in retrieving the same documents.<n>The loss penalizes discrepancies between the top-k ranked documents retrieved for diverse but semantic equivalent queries.
  arXiv  Detail & Related papers  (2025-08-11T13:34:59Z)
• DREAM: Document Reconstruction via End-to-end Autoregressive Model [53.51754520966657]
  We present an innovative autoregressive model specifically designed for document reconstruction, referred to as Document Reconstruction via End-to-end Autoregressive Model (DREAM)<n>We establish a standardized definition of the document reconstruction task, and introduce a novel Document Similarity Metric (DSM) and DocRec1K dataset for assessing the performance of the task.
  arXiv  Detail & Related papers  (2025-07-08T09:24:07Z)
• Advancing Question Answering on Handwritten Documents: A State-of-the-Art Recognition-Based Model for HW-SQuAD [30.559280110711143]
  This paper proposes a novel recognition-based approach that improves upon the previous state-of-the-art on the HW-SQuAD and BenthamQA datasets. Our model incorporates transformer-based document retrieval and ensemble methods at the model level, achieving an Exact Match score of 82.02% and 69% in HW-SQuAD and BenthamQA datasets.
  arXiv  Detail & Related papers  (2024-06-25T10:18:50Z)
• DocGenome: An Open Large-scale Scientific Document Benchmark for Training and Testing Multi-modal Large Language Models [63.466265039007816]
  We present DocGenome, a structured document benchmark constructed by annotating 500K scientific documents from 153 disciplines in the arXiv open-access community. We conduct extensive experiments to demonstrate the advantages of DocGenome and objectively evaluate the performance of large models on our benchmark.
  arXiv  Detail & Related papers  (2024-06-17T15:13:52Z)
• Knowledge-Centric Templatic Views of Documents [2.654058995940072]
  Authors often share their ideas in various document formats, such as slide decks, newsletters, reports, and posters. We introduce a novel unified evaluation framework that can be adapted to measuring the quality of document generators. We conduct a human evaluation, which shows that people prefer 82% of the documents generated with our method.
  arXiv  Detail & Related papers  (2024-01-13T01:22:15Z)
• PART: Pre-trained Authorship Representation Transformer [52.623051272843426]
  Authors writing documents imprint identifying information within their texts.<n>Previous works use hand-crafted features or classification tasks to train their authorship models.<n>We propose a contrastively trained model fit to learn textbfauthorship embeddings instead of semantics.
  arXiv  Detail & Related papers  (2022-09-30T11:08:39Z)
• Bi-VLDoc: Bidirectional Vision-Language Modeling for Visually-Rich Document Understanding [88.88844606781987]
  Multi-modal document pre-trained models have proven to be very effective in a variety of visually-rich document understanding (VrDU) tasks.<n>The way they model and exploit the interactions between vision and language on documents has hindered them from better generalization ability and higher accuracy.<n>In this work, we investigate the problem of vision-language joint representation learning for VrDU mainly from the perspective of supervisory signals.
  arXiv  Detail & Related papers  (2022-06-27T09:58:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.