Leveraging Graph Structure in Seq2Seq Models for Knowledge Graph Link Prediction

• URL: http://arxiv.org/abs/2605.18211v1
• Date: Mon, 18 May 2026 10:56:14 GMT
• Title: Leveraging Graph Structure in Seq2Seq Models for Knowledge Graph Link Prediction
• Abstract summary: Graph-Augmented Sequence-to-Sequence (GA-S2S)<n>GA-S2S is a novel framework that integrates a T5-decoder with a Graph Attention Network (RGAT)<n>Preliminary experiments on the CoEx dataset demonstrate that GA-S2S outperforms competitive Seq2Seq-based baseline models.
• Score: 18.96090071874467
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We introduce Graph-Augmented Sequence-to-Sequence (GA-S2S), a novel framework that integrates a T5-small encoder-decoder with a Relational Graph Attention Network (RGAT) to improve link prediction in knowledge graphs. While existing Seq2Seq models rely solely on surface-level textual descriptions of entities and relations and at best, flatten the neighborhoods of a query entity into a single linear sequence, thereby discarding the inherent graph structure, GA-S2S jointly encodes both textual features and the full $k$-hop subgraph topology surrounding the query entity. By integrating raw encoder outputs with RGAT's relation-aware embeddings, our model captures and leverages richer multi-hop relational patterns and textual information. Our preliminary experiments on the CoDEx dataset demonstrate that GA-S2S outperforms competitive Seq2Seq-based baseline models, achieving up to a 19\% relative gain in link prediction accuracy.

Related papers

• Semi-supervised Instruction Tuning for Large Language Models on Text-Attributed Graphs [62.544129365882014]
  We propose a novel Semi-supervised Instruction Tuning pipeline for Graph Learning, named SIT-Graph.<n> SIT-Graph is model-agnostic and can be seamlessly integrated into any graph instruction tuning method that utilizes LLMs as the predictor.<n>Extensive experiments demonstrate that when incorporated into state-of-the-art graph instruction tuning methods, SIT-Graph significantly enhances their performance on text-attributed graph benchmarks.
  arXiv  Detail & Related papers  (2026-01-19T08:10:53Z)
• Youtu-GraphRAG: Vertically Unified Agents for Graph Retrieval-Augmented Complex Reasoning [32.78218766121055]
  Graph retrieval-augmented generation (GraphRAG) has effectively enhanced large language models in complex reasoning.<n>We propose a vertically unified agentic paradigm, Youtu-GraphRAG, to jointly connect the entire framework as an intricate integration.
  arXiv  Detail & Related papers  (2025-08-27T13:13:20Z)
• Context Pooling: Query-specific Graph Pooling for Generic Inductive Link Prediction in Knowledge Graphs [55.918039693545616]
  We introduce a novel method, named Context Pooling, to enhance GNN-based models' efficacy for link predictions in Knowledge Graphs.<n>Our method is generic and assessed by being applied to two state-of-the-art (SOTA) models on three public transductive and inductive datasets.
  arXiv  Detail & Related papers  (2025-07-10T09:54:37Z)
• GraphRAG-Causal: A novel graph-augmented framework for causal reasoning and annotation in news [0.48379436447082086]
  GraphRAG-Causal combines graph-based retrieval with large language models to enhance causal reasoning in news analysis.<n>It achieves an impressive F1-score of 82.1% on causal classification using just 20 few-shot examples.<n>This approach significantly boosts accuracy and consistency, making it highly suitable for real-time applications in news reliability assessment, misinformation detection, and policy analysis.
  arXiv  Detail & Related papers  (2025-06-13T09:09:08Z)
• A Pure Transformer Pretraining Framework on Text-attributed Graphs [50.833130854272774]
  We introduce a feature-centric pretraining perspective by treating graph structure as a prior. Our framework, Graph Sequence Pretraining with Transformer (GSPT), samples node contexts through random walks. GSPT can be easily adapted to both node classification and link prediction, demonstrating promising empirical success on various datasets.
  arXiv  Detail & Related papers  (2024-06-19T22:30:08Z)
• Article Classification with Graph Neural Networks and Multigraphs [0.12499537119440243]
  We propose a method to enhance the performance of article classification by enriching simple Graph Neural Network (GNN) pipelines with multi-graph representations. fully supervised transductive node classification experiments are conducted on the Open Graph Benchmark OGBN-arXiv dataset and the PubMed diabetes dataset. Results demonstrate that multi-graphs consistently improve the performance of a variety of GNN models compared to the default graphs.
  arXiv  Detail & Related papers  (2023-09-20T14:18:04Z)
• GrannGAN: Graph annotation generative adversarial networks [72.66289932625742]
  We consider the problem of modelling high-dimensional distributions and generating new examples of data with complex relational feature structure coherent with a graph skeleton. The model we propose tackles the problem of generating the data features constrained by the specific graph structure of each data point by splitting the task into two phases. In the first it models the distribution of features associated with the nodes of the given graph, in the second it complements the edge features conditionally on the node features.
  arXiv  Detail & Related papers  (2022-12-01T11:49:07Z)
• Question-Answer Sentence Graph for Joint Modeling Answer Selection [122.29142965960138]
  We train and integrate state-of-the-art (SOTA) models for computing scores between question-question, question-answer, and answer-answer pairs. Online inference is then performed to solve the AS2 task on unseen queries.
  arXiv  Detail & Related papers  (2022-02-16T05:59:53Z)
• Compositionality-Aware Graph2Seq Learning [2.127049691404299]
  compositionality in a graph can be associated to the compositionality in the output sequence in many graph2seq tasks. We adopt the multi-level attention pooling (MLAP) architecture, that can aggregate graph representations from multiple levels of information localities. We demonstrate that the model having the MLAP architecture outperform the previous state-of-the-art model with more than seven times fewer parameters.
  arXiv  Detail & Related papers  (2022-01-28T15:22:39Z)
• Jointly Cross- and Self-Modal Graph Attention Network for Query-Based Moment Localization [77.21951145754065]
  We propose a novel Cross- and Self-Modal Graph Attention Network (CSMGAN) that recasts this task as a process of iterative messages passing over a joint graph. Our CSMGAN is able to effectively capture high-order interactions between two modalities, thus enabling a further precise localization.
  arXiv  Detail & Related papers  (2020-08-04T08:25:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.