FRAGMENT: Factorized Graph Representations for Document Generation and Editing via Entity-Aware Transformations

• URL: http://arxiv.org/abs/2608.18679v1
• Date: Wed, 19 Aug 2026 08:29:19 GMT
• Title: FRAGMENT: Factorized Graph Representations for Document Generation and Editing via Entity-Aware Transformations
• Abstract summary: Generative models operating at the pixel or token level often struggle to capture spatial layout, textual content, and logical structure dependencies effectively.<n>We explore FRAGMENT, a generative framework that represents a document as a typed relational graph and factorizes its distribution as p(structure, content) = p(structure) * p(content | structure)<n>Experiments on DocLayNet, FUNSD, and SROIE evaluate FRAGMENT alongside representative autoregressive, layout-only, and graph-based baselines.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Structured documents such as invoices, forms, reports, and scientific articles derive meaning from the interplay between spatial layout, textual content, and logical structure. Generative models operating at the pixel or token level often struggle to capture these dependencies effectively. We explore FRAGMENT, a generative framework that represents a document as a typed relational graph and factorizes its distribution as p(structure, content) = p(structure) * p(content | structure). The framework consists of two stages. The first stage, the Architect, is a causally masked Transformer conditioned on document category that autoregressively generates the graph topology and typed spatial relations. The second stage, the Builder, is a GATv2-based graph attention network that enriches the graph with normalized bounding boxes, text, and visual style attributes. Both stages define explicit likelihood models, yielding a tractable document-level likelihood that serves as an anomaly score for forgery detection. For controlled editing, a prompt-conditioned extension injects instruction embeddings into the Builder through cross-attention, enabling semantic and entity-aware modifications. We describe training on DocLayNet and fine-tuning on FUNSD and SROIE. Experiments on DocLayNet, FUNSD, and SROIE evaluate FRAGMENT alongside representative autoregressive, layout-only, and graph-based baselines, providing an empirical analysis of the characteristics and trade-offs of the proposed factorized graph generation framework.

Related papers

• Semantic-Structural Alignment for Generative Pictorial Charts [59.45629259524998]
  We present a generative framework for the automated synthesis of pictorial charts.<n>We frame the problem as a dual-conditioned generation task guided by two parallel external control signals.<n>Our method produces pictorial charts that are both artistically compelling and structurally consistent.
  arXiv  Detail & Related papers  (2026-05-05T05:20:46Z)
• Graph Construction and Matching for Imperative Programs using Neural and Structural Methods [0.0]
  We present a pipeline that converts imperative programs and their annotations into typed, attributed graphs.<n>Our experiments cover datasets including C with ACSL, Java with JML, and Dafny for C#.
  arXiv  Detail & Related papers  (2026-04-29T11:59:53Z)
• Hierarchical Graph Topic Modeling with Topic Tree-based Transformer [35.554979581137296]
  We propose a Hierarchical Graph Topic Modeling Transformer to integrate both topic hierarchy within documents and graph hierarchy across documents.<n>To preserve both topic and graph hierarchies, we design our model in hyperbolic space and propose Hyperbolic Doubly Recurrent Neural Network.<n>Both supervised and unsupervised experiments verify the effectiveness of our model.
  arXiv  Detail & Related papers  (2025-02-17T01:55:29Z)
• A Pure Transformer Pretraining Framework on Text-attributed Graphs [50.833130854272774]
  We introduce a feature-centric pretraining perspective by treating graph structure as a prior. Our framework, Graph Sequence Pretraining with Transformer (GSPT), samples node contexts through random walks. GSPT can be easily adapted to both node classification and link prediction, demonstrating promising empirical success on various datasets.
  arXiv  Detail & Related papers  (2024-06-19T22:30:08Z)
• A Semantic Mention Graph Augmented Model for Document-Level Event Argument Extraction [12.286432133599355]
  Document-level Event Argument Extraction (DEAE) aims to identify arguments and their specific roles from an unstructured document. advanced approaches on DEAE utilize prompt-based methods to guide pre-trained language models (PLMs) in extracting arguments from input documents. We propose a semantic mention Graph Augmented Model (GAM) to address these two problems in this paper.
  arXiv  Detail & Related papers  (2024-03-12T08:58:07Z)
• GraphKD: Exploring Knowledge Distillation Towards Document Object Detection with Structured Graph Creation [14.511401955827875]
  Object detection in documents is a key step to automate the structural elements identification process. We present a graph-based knowledge distillation framework to correctly identify and localize the document objects in a document image.
  arXiv  Detail & Related papers  (2024-02-17T23:08:32Z)
• Enhancing Visually-Rich Document Understanding via Layout Structure Modeling [91.07963806829237]
  We propose GraphLM, a novel document understanding model that injects layout knowledge into the model. We evaluate our model on various benchmarks, including FUNSD, XFUND and CORD, and achieve state-of-the-art results.
  arXiv  Detail & Related papers  (2023-08-15T13:53:52Z)
• Doc2SoarGraph: Discrete Reasoning over Visually-Rich Table-Text Documents via Semantic-Oriented Hierarchical Graphs [79.0426838808629]
  We propose TAT-DQA, i.e. to answer the question over a visually-rich table-text document. Specifically, we propose a novel Doc2SoarGraph framework with enhanced discrete reasoning capability. We conduct extensive experiments on TAT-DQA dataset, and the results show that our proposed framework outperforms the best baseline model by 17.73% and 16.91% in terms of Exact Match (EM) and F1 score respectively on the test set.
  arXiv  Detail & Related papers  (2023-05-03T07:30:32Z)
• Entity Type Prediction Leveraging Graph Walks and Entity Descriptions [4.147346416230273]
  textitGRAND is a novel approach for entity typing leveraging different graph walk strategies in RDF2vec together with textual entity descriptions. The proposed approach outperforms the baseline approaches on the benchmark datasets DBpedia and FIGER for entity typing in KGs for both fine-grained and coarse-grained classes.
  arXiv  Detail & Related papers  (2022-07-28T13:56:55Z)
• Iterative Scene Graph Generation [55.893695946885174]
  Scene graph generation involves identifying object entities and their corresponding interaction predicates in a given image (or video) Existing approaches to scene graph generation assume certain factorization of the joint distribution to make the estimation iteration feasible. We propose a novel framework that addresses this limitation, as well as introduces dynamic conditioning on the image.
  arXiv  Detail & Related papers  (2022-07-27T10:37:29Z)
• FactGraph: Evaluating Factuality in Summarization with Semantic Graph Representations [114.94628499698096]
  We propose FactGraph, a method that decomposes the document and the summary into structured meaning representations (MRs) MRs describe core semantic concepts and their relations, aggregating the main content in both document and summary in a canonical form, and reducing data sparsity. Experiments on different benchmarks for evaluating factuality show that FactGraph outperforms previous approaches by up to 15%.
  arXiv  Detail & Related papers  (2022-04-13T16:45:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.