Scaling Multi-Hop Training Data via Graph-Constrained Path Selection

• URL: http://arxiv.org/abs/2605.31238v1
• Date: Fri, 29 May 2026 12:39:03 GMT
• Title: Scaling Multi-Hop Training Data via Graph-Constrained Path Selection
• Abstract summary: We decouple the two operations: reasoning paths are enumerated offline over a graph of contextual keyword centroids, and the teacher is invoked only to pre-validated paths.<n>At equal training scale, constrained and unconstrained chains yield indistinguishable downstream performance, and the gain at full scale comes from a 4.4$times$ expansion of the usable corpus.<n>Fine-tuning Qwen3-32B on 80K examples constructed from the CUAD legal contract corpus improves closed-book Token F1 from 21.66% to 38.58%.
• Score: 40.133031562641044
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Endowing large language models with compositional reasoning over specialized documents requires multi-hop training data at scale, where such data rarely exists outside of curated benchmarks built on structured sources. To construct it directly from plain, unannotated text, existing methods ask a single teacher model to jointly discover an evidence path through a document and verbalize it as a question-answer pair. However, these methods degrade sharply when documents are structured around repetitive templates and densely cross-referencing clauses, conditions that characterize most real-world specialized corpora. In this work, we decouple the two operations: reasoning paths are enumerated offline over a graph of contextual keyword centroids, and the teacher is invoked only to verbalize pre-validated paths. The graph enforces five geometric admissibility constraints, for which we provide Gram-matrix arguments establishing that local similarity bounds alone admit endpoint drift up to ${\sim}91^{\circ}$, and that an upper similarity bound is necessary to exit dense embedding cliques formed by boilerplate text. A matched-size ablation isolates the mechanism: at equal training scale, constrained and unconstrained chains yield indistinguishable downstream performance, and the gain at full scale comes from a 4.4$\times$ expansion of the usable corpus rather than from higher per-chain quality -- reframing the role of graph constraints, in this setting, as raising teacher synthesizability rather than improving chain content. Fine-tuning Qwen3-32B on 80K examples constructed from the CUAD legal contract corpus improves closed-book Token F1 from 21.66% to 38.58%. We have released our codes at https://github.com/hkgai-official/GCSCS.

Related papers

• A Training-Free, Alignment-Free Approach to Corporate Intelligence: Application to SEC Filings [0.0]
  We present a training-free, alignment-free framework for corporate intelligence built on deterministic sparse seed vectors.<n>Hashing word strings into a fixed high-dimensional basis places all documents and all temporal epochs in a common coordinate system by construction, removing any need for training or alignment.<n>We show how material corporate events, among them Boeing's 737 MAX crisis, Intel's supply-chain disruptions, and Bunge's acquisition of Viterra, emerge as distinct, interpretable semantic profiles.
  arXiv  Detail & Related papers  (2026-09-10T14:32:35Z)
• Weave of Formal Thought [51.56484100374058]
  We introduce Weave of Formal Thought (WoFT), a paradigm uniting rigorous syntactic validation with learned structural representations.<n>We present a latent-variable fine-tuning method training the language model to interleave non-terminal grammar symbols directly into generation.<n>For Python, fine-tuning StarCoder2-3B with our RWS objective reduces per-token cross-entropy by 14.3% relative to a text-only SFT baseline.
  arXiv  Detail & Related papers  (2026-06-24T15:58:11Z)
• Tokenization, Fusion and Decoupling: Bridging the Granularity Mismatch Between Large Language Models and Knowledge Graphs [20.946228883628013]
  We propose KGT, a novel framework that uses dedicated entity tokens to enable efficient, full-space prediction.<n>We first introduce specialized tokenization to construct feature representations at the level of dedicated entity tokens.<n>We then fuse pre-trained structural and textual features into these unified embeddings via a relation-guided gating mechanism.
  arXiv  Detail & Related papers  (2026-02-26T07:20:40Z)
• <SOG_k>: One LLM Token for Explicit Graph Structural Understanding [57.017902343605364]
  We propose to incorporate one special token SOG_k> to fully represent the Structure Of Graph within a unified token space.<n>SOG_k> empowers LLMs to understand, generate, and reason in a concise and accurate manner.
  arXiv  Detail & Related papers  (2026-02-02T07:55:09Z)
• $G^2$-Reader: Dual Evolving Graphs for Multimodal Document QA [53.491241153213565]
  $G2$-Reader is a dual-graph system for multimodal question answering.<n>$G2$-Reader with Qwen3-VL-32B-Instruct reaches 66.21% average accuracy, outperforming strong baselines and a standalone GPT-5 (53.08%)<n>On VisDoMBench across five multimodal domains, $G2$-Reader with Qwen3-VL-32B-Instruct reaches 66.21% average accuracy, outperforming strong baselines and a standalone GPT-5 (53.08%)
  arXiv  Detail & Related papers  (2026-01-29T17:52:54Z)
• Semi-supervised Instruction Tuning for Large Language Models on Text-Attributed Graphs [62.544129365882014]
  We propose a novel Semi-supervised Instruction Tuning pipeline for Graph Learning, named SIT-Graph.<n> SIT-Graph is model-agnostic and can be seamlessly integrated into any graph instruction tuning method that utilizes LLMs as the predictor.<n>Extensive experiments demonstrate that when incorporated into state-of-the-art graph instruction tuning methods, SIT-Graph significantly enhances their performance on text-attributed graph benchmarks.
  arXiv  Detail & Related papers  (2026-01-19T08:10:53Z)
• N2N-GQA: Noise-to-Narrative for Graph-Based Table-Text Question Answering Using LLMs [0.0]
  Multi-hop question answering over hybrid table-text data requires retrieving and reasoning across multiple evidence pieces from large corpora.<n>Standard Retrieval-Augmented Generation (RAG) pipelines process documents as flat ranked lists, causing retrieval noise to obscure reasoning chains.<n>N2N-GQA is the first zeroshot framework for open-domain hybrid table-text QA that constructs dynamic evidence graphs from noisy retrieval outputs.
  arXiv  Detail & Related papers  (2026-01-10T15:55:15Z)
• Few-Shot Connectivity-Aware Text Line Segmentation in Historical Documents [1.4065611645922207]
  In this work, we demonstrate that small and simple architectures, coupled with a topology-aware loss function, are more accurate and data-efficient than more complex alternatives.<n>Our methodology significantly improves upon the current state-of-the-art on the U-DIADS-TL dataset, with a 200% increase in Recognition Accuracy and a 75% increase in Line Intersection over Union.
  arXiv  Detail & Related papers  (2025-08-26T16:11:32Z)
• Token-Level Graphs for Short Text Classification [1.6819960041696331]
  We propose an approach which constructs text graphs entirely based on tokens obtained through pre-trained language models (PLMs)<n>Our method captures contextual and semantic information, overcomes vocabulary constraints, and allows for context-dependent word meanings.<n> Experimental results demonstrate how our method consistently achieves higher scores or on-par performance with existing methods.
  arXiv  Detail & Related papers  (2024-12-17T10:19:44Z)
• Match me if you can: Semi-Supervised Semantic Correspondence Learning with Unpaired Images [76.47980643420375]
  This paper builds on the hypothesis that there is an inherent data-hungry matter in learning semantic correspondences. We demonstrate a simple machine annotator reliably enriches paired key points via machine supervision. Our models surpass current state-of-the-art models on semantic correspondence learning benchmarks like SPair-71k, PF-PASCAL, and PF-WILLOW.
  arXiv  Detail & Related papers  (2023-11-30T13:22:15Z)
• Single Sequence Prediction over Reasoning Graphs for Multi-hop QA [8.442412179333205]
  We propose a single-sequence prediction method over a local reasoning graph (model)footnoteCode/Models. We use a graph neural network to encode this graph structure and fuse the resulting representations into the entity representations of the model. Our experiments show significant improvements in answer exact-match/F1 scores and faithfulness of grounding in the reasoning path.
  arXiv  Detail & Related papers  (2023-07-01T13:15:09Z)
• Structured Dialogue Discourse Parsing [79.37200787463917]
  discourse parsing aims to uncover the internal structure of a multi-participant conversation. We propose a principled method that improves upon previous work from two perspectives: encoding and decoding. Experiments show that our method achieves new state-of-the-art, surpassing the previous model by 2.3 on STAC and 1.5 on Molweni.
  arXiv  Detail & Related papers  (2023-06-26T22:51:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.