HyperG: Hypergraph-Enhanced LLMs for Structured Knowledge

• URL: http://arxiv.org/abs/2502.18125v1
• Date: Tue, 25 Feb 2025 11:47:32 GMT
• Title: HyperG: Hypergraph-Enhanced LLMs for Structured Knowledge
• Authors: Sirui Huang, Hanqian Li, Yanggan Gu, Xuming Hu, Qing Li, Guandong Xu,
• Abstract summary: HyperG is a hypergraph-based generation framework aimed at enhancing Large Language Models' ability to process structured knowledge.<n>Specifically, HyperG first augments sparse data with contextual information, and incorporate a prompt-attentive hypergraph learning network to encode both the augmented information and the intricate structural relationships within the data.<n>To validate the effectiveness and generalization of HyperG, we conduct extensive experiments across two different downstream tasks requiring structured knowledge.
• Score: 25.279158571663036
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Given that substantial amounts of domain-specific knowledge are stored in structured formats, such as web data organized through HTML, Large Language Models (LLMs) are expected to fully comprehend this structured information to broaden their applications in various real-world downstream tasks. Current approaches for applying LLMs to structured data fall into two main categories: serialization-based and operation-based methods. Both approaches, whether relying on serialization or using SQL-like operations as an intermediary, encounter difficulties in fully capturing structural relationships and effectively handling sparse data. To address these unique characteristics of structured data, we propose HyperG, a hypergraph-based generation framework aimed at enhancing LLMs' ability to process structured knowledge. Specifically, HyperG first augment sparse data with contextual information, leveraging the generative power of LLMs, and incorporate a prompt-attentive hypergraph learning (PHL) network to encode both the augmented information and the intricate structural relationships within the data. To validate the effectiveness and generalization of HyperG, we conduct extensive experiments across two different downstream tasks requiring structured knowledge.

Related papers

• $\texttt{InfoHier}$: Hierarchical Information Extraction via Encoding and Embedding [0.7499722271664147]
  $texttInfoHier$ is a framework for learning robust latent representations and hierarchical structures. It provides adaptive representations, enhancing HC's ability to capture complex patterns. It integrates HC loss to refine SSL training, resulting in representations more attuned to the underlying information hierarchy.
  arXiv  Detail & Related papers  (2025-01-15T10:58:32Z)
• Knowledge prompt chaining for semantic modeling [0.0]
  We propose a novel automatic semantic modeling framework: Knowledge Prompt Chaining. It canserialize the graph-structured knowledge and inject it into the LLMs properly. Based on experimental results, our method achieves better performance than existing leading techniques.
  arXiv  Detail & Related papers  (2025-01-15T03:00:57Z)
• Filter-then-Generate: Large Language Models with Structure-Text Adapter for Knowledge Graph Completion [20.973071287301067]
  Large Language Models (LLMs) present massive inherent knowledge and superior semantic comprehension capability. Empirical evidence suggests that LLMs consistently perform worse than conventional knowledge graph completion approaches. We propose a novel instruction-tuning-based method, namely FtG, to address these challenges.
  arXiv  Detail & Related papers  (2024-12-12T09:22:04Z)
• LLaSA: Large Language and Structured Data Assistant [27.452536284165273]
  Graph Neutral Networks (GNNs) have been introduced as an additional modality into the input of Large Language Models (LLMs)<n>We propose textbfLarge textbfLanguage and textbfStructured Data textbfAssistant (LLaSA) to enhance LLMs' ability to handle structured data.
  arXiv  Detail & Related papers  (2024-11-16T12:27:14Z)
• Web-Scale Visual Entity Recognition: An LLM-Driven Data Approach [56.55633052479446]
  Web-scale visual entity recognition presents significant challenges due to the lack of clean, large-scale training data. We propose a novel methodology to curate such a dataset, leveraging a multimodal large language model (LLM) for label verification, metadata generation, and rationale explanation. Experiments demonstrate that models trained on this automatically curated data achieve state-of-the-art performance on web-scale visual entity recognition tasks.
  arXiv  Detail & Related papers  (2024-10-31T06:55:24Z)
• StructRAG: Boosting Knowledge Intensive Reasoning of LLMs via Inference-time Hybrid Information Structurization [94.31508613367296]
  Retrieval-augmented generation (RAG) is a key means to effectively enhance large language models (LLMs) We propose StructRAG, which can identify the optimal structure type for the task at hand, reconstruct original documents into this structured format, and infer answers based on the resulting structure. Experiments show that StructRAG achieves state-of-the-art performance, particularly excelling in challenging scenarios.
  arXiv  Detail & Related papers  (2024-10-11T13:52:44Z)
• Struct-X: Enhancing Large Language Models Reasoning with Structured Data [38.558614152006975]
  Struct-X operates through five key phases: read-model-fill-reflect-reason'' It encodes structured data into a topological space using graph embeddings. It fills in missing entity information with knowledge retrieval modules. The final phase involves constructing a topological network with selected tokens.
  arXiv  Detail & Related papers  (2024-07-17T13:06:25Z)
• Think-on-Graph 2.0: Deep and Faithful Large Language Model Reasoning with Knowledge-guided Retrieval Augmented Generation [14.448198170932226]
  Think-on-Graph 2.0 (ToG-2) is a hybrid RAG framework that iteratively retrieves information from both unstructured and structured knowledge sources.<n>ToG-2 alternates between graph retrieval and context retrieval to search for in-depth clues relevant to the question.<n>It achieves overall state-of-the-art (SOTA) performance on 6 out of 7 knowledge-intensive datasets with GPT-3.5.
  arXiv  Detail & Related papers  (2024-07-15T15:20:40Z)
• DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text [73.68051228972024]
  Large Language Models (LLMs) have exhibited impressive generation capabilities, but they suffer from hallucinations when relying on their internal knowledge. Retrieval-augmented LLMs have emerged as a potential solution to ground LLMs in external knowledge.
  arXiv  Detail & Related papers  (2023-10-31T04:37:57Z)
• Harnessing Explanations: LLM-to-LM Interpreter for Enhanced Text-Attributed Graph Representation Learning [51.90524745663737]
  A key innovation is our use of explanations as features, which can be used to boost GNN performance on downstream tasks. Our method achieves state-of-the-art results on well-established TAG datasets. Our method significantly speeds up training, achieving a 2.88 times improvement over the closest baseline on ogbn-arxiv.
  arXiv  Detail & Related papers  (2023-05-31T03:18:03Z)
• StructGPT: A General Framework for Large Language Model to Reason over Structured Data [117.13986738340027]
  We develop an emphIterative Reading-then-Reasoning(IRR) approach for solving question answering tasks based on structured data. Our approach can significantly boost the performance of ChatGPT and achieve comparable performance against the full-data supervised-tuning baselines.
  arXiv  Detail & Related papers  (2023-05-16T17:45:23Z)
• LasUIE: Unifying Information Extraction with Latent Adaptive Structure-aware Generative Language Model [96.889634747943]
  Universally modeling all typical information extraction tasks (UIE) with one generative language model (GLM) has revealed great potential. We propose a novel structure-aware GLM, fully unleashing the power of syntactic knowledge for UIE. Over 12 IE benchmarks across 7 tasks our system shows significant improvements over the baseline UIE system.
  arXiv  Detail & Related papers  (2023-04-13T04:01:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.