LLMs Underperform Graph-Based Parsers on Supervised Relation Extraction for Complex Graphs

• URL: http://arxiv.org/abs/2604.08752v1
• Date: Thu, 09 Apr 2026 20:34:33 GMT
• Title: LLMs Underperform Graph-Based Parsers on Supervised Relation Extraction for Complex Graphs
• Abstract summary: Large language models (LLMs) have been adopted as a promising tool for relation extraction.<n>We show that their performance still lags behind much smaller architectures when the linguistic graph underlying a text has great complexity.
• Score: 12.09105878208262
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Relation extraction represents a fundamental component in the process of creating knowledge graphs, among other applications. Large language models (LLMs) have been adopted as a promising tool for relation extraction, both in supervised and in-context learning settings. However, in this work we show that their performance still lags behind much smaller architectures when the linguistic graph underlying a text has great complexity. To demonstrate this, we evaluate four LLMs against a graph-based parser on six relation extraction datasets with sentence graphs of varying sizes and complexities. Our results show that the graph-based parser increasingly outperforms the LLMs, as the number of relations in the input documents increases. This makes the much lighter graph-based parser a superior choice in the presence of complex linguistic graphs.

Related papers

• GraphSeek: Next-Generation Graph Analytics with LLMs [15.668202711555749]
  LLMs promise accessible natural language (NL) graph analytics, but they fail to process industry-scale property graphs effectively and efficiently.<n>We devise a novel abstraction for complex multi-query analytics over such graphs.<n>We use this abstraction as the basis of the first LLM-enhanced graph analytics framework called GraphSeek.
  arXiv  Detail & Related papers  (2026-02-11T17:20:06Z)
• Colorful Talks with Graphs: Human-Interpretable Graph Encodings for Large Language Models [12.496005049442319]
  Graph problems are fundamentally challenging for large language models (LLMs)<n>We introduce a human-interpretable structural encoding strategy for graph-to-text translation.<n>Our method enhances LLM performance especially on graph tasks that require reasoning over global graph structure.
  arXiv  Detail & Related papers  (2026-02-11T00:15:29Z)
• GRIL: Knowledge Graph Retrieval-Integrated Learning with Large Language Models [59.72897499248909]
  We propose a novel graph retriever trained end-to-end with Large Language Models (LLMs)<n>Within the extracted subgraph, structural knowledge and semantic features are encoded via soft tokens and the verbalized graph, respectively, which are infused into the LLM together.<n>Our approach consistently achieves state-of-the-art performance, validating the strength of joint graph-LLM optimization for complex reasoning tasks.
  arXiv  Detail & Related papers  (2025-09-20T02:38:00Z)
• ZOGRASCOPE: A New Benchmark for Semantic Parsing over Property Graphs [3.0748861313823]
  Property graphs (PGs) have seen increased adoption as a means of representing complex structured information.<n>Despite their growing popularity in industry, PGs remain relatively underrepresented in semantic parsing research.<n>We introduce ZOGRASCOPE, a benchmark designed specifically for PGs and queries written in Cypher.
  arXiv  Detail & Related papers  (2025-03-07T09:33:30Z)
• GUNDAM: Aligning Large Language Models with Graph Understanding [10.080136100700692]
  We introduce the textbfGraph textbfUnderstanding for textbfNatural Language textbfDriven textbfAnalytical textbfModel (model) This model adapts LLMs to better understand and engage with the structure of graph data, enabling them to perform complex reasoning tasks by leveraging the graph's structure itself.
  arXiv  Detail & Related papers  (2024-09-30T07:59:10Z)
• Hierarchical Compression of Text-Rich Graphs via Large Language Models [63.75293588479027]
  Text-rich graphs are prevalent in data mining contexts like e-commerce and academic graphs. This paper introduces Hierarchical Compression'' (HiCom), a novel method to align the capabilities of LLMs with the structure of text-rich graphs. HiCom can outperform both GNNs and LLM backbones for node classification on e-commerce and citation graphs.
  arXiv  Detail & Related papers  (2024-06-13T07:24:46Z)
• Graph Chain-of-Thought: Augmenting Large Language Models by Reasoning on Graphs [60.71360240206726]
  Large language models (LLMs) suffer from hallucinations, especially on knowledge-intensive tasks. Existing works propose to augment LLMs with individual text units retrieved from external knowledge corpora. We propose a framework called Graph Chain-of-thought (Graph-CoT) to augment LLMs with graphs by encouraging LLMs to reason on the graph iteratively.
  arXiv  Detail & Related papers  (2024-04-10T15:41:53Z)
• LLaGA: Large Language and Graph Assistant [73.71990472543027]
  Large Language and Graph Assistant (LLaGA) is an innovative model to handle the complexities of graph-structured data. LLaGA excels in versatility, generalizability and interpretability, allowing it to perform consistently well across different datasets and tasks. Our experiments show that LLaGA delivers outstanding performance across four datasets and three tasks using one single model.
  arXiv  Detail & Related papers  (2024-02-13T02:03:26Z)
• Large Language Models on Graphs: A Comprehensive Survey [77.16803297418201]
  We provide a systematic review of scenarios and techniques related to large language models on graphs. We first summarize potential scenarios of adopting LLMs on graphs into three categories, namely pure graphs, text-attributed graphs, and text-paired graphs. We discuss the real-world applications of such methods and summarize open-source codes and benchmark datasets.
  arXiv  Detail & Related papers  (2023-12-05T14:14:27Z)
• Integrating Graphs with Large Language Models: Methods and Prospects [68.37584693537555]
  Large language models (LLMs) have emerged as frontrunners, showcasing unparalleled prowess in diverse applications. Merging the capabilities of LLMs with graph-structured data has been a topic of keen interest. This paper bifurcates such integrations into two predominant categories.
  arXiv  Detail & Related papers  (2023-10-09T07:59:34Z)
• Beyond Text: A Deep Dive into Large Language Models' Ability on Understanding Graph Data [13.524529952170672]
  Large language models (LLMs) have achieved impressive performance on many natural language processing tasks. We aim to assess whether LLMs can effectively process graph data and leverage topological structures to enhance performance. By comparing LLMs' performance with specialized graph models, we offer insights into the strengths and limitations of employing LLMs for graph analytics.
  arXiv  Detail & Related papers  (2023-10-07T23:25:22Z)
• Talk like a Graph: Encoding Graphs for Large Language Models [15.652881653332194]
  We study the first comprehensive study of encoding graph-structured data as text for consumption by large language models (LLMs) We show that LLM performance on graph reasoning tasks varies on three fundamental levels: (1) the graph encoding method, (2) the nature of the graph task itself, and (3) interestingly, the very structure of the graph considered.
  arXiv  Detail & Related papers  (2023-10-06T19:55:21Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.