GraphIC: A Graph-Based In-Context Example Retrieval Model for Multi-Step Reasoning

• URL: http://arxiv.org/abs/2410.02203v3
• Date: Tue, 25 Feb 2025 03:10:28 GMT
• Title: GraphIC: A Graph-Based In-Context Example Retrieval Model for Multi-Step Reasoning
• Authors: Jiale Fu, Yaqing Wang, Simeng Han, Jiaming Fan, Xu Yang,
• Abstract summary: In-context learning (ICL) enhances large language models (LLMs) by incorporating demonstration examples.<n>Current methods typically use text embeddings to measure semantic similarity, which often introduces bias in multi-step reasoning tasks.<n>We propose GraphIC, a graph-based retrieval model that leverages reasoning-aware representation and specialized similarity metric for in-context example retrieval.
• Score: 22.115256310400817
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In-context learning (ICL) enhances large language models (LLMs) by incorporating demonstration examples, yet its effectiveness heavily depends on the quality of selected examples. Current methods typically use text embeddings to measure semantic similarity, which often introduces bias in multi-step reasoning tasks. This occurs because text embeddings contain irrelevant semantic information and lack deeper reasoning structures. To address this, we propose GraphIC, a graph-based retrieval model that leverages reasoning-aware representation and specialized similarity metric for in-context example retrieval. GraphIC first constructs thought graphs-directed, node-attributed graphs that explicitly model reasoning steps and their dependencies-for candidate examples and queries. This approach filters out superficial semantics while preserving essential reasoning processes. Next, GraphIC retrieves examples using a novel similarity metric tailored for these graphs, capturing sequential reasoning patterns and asymmetry between examples. Comprehensive evaluations across mathematical reasoning, code generation, and logical reasoning tasks demonstrate that GraphIC outperforms 10 baseline methods. Our results highlight the importance of reasoning-aware retrieval in ICL, offering a robust solution for enhancing LLM performance in multi-step reasoning scenarios.

Related papers

• Measuring Similarity in Causal Graphs: A Framework for Semantic and Structural Analysis [0.7373617024876725]
  Causal graphs are commonly used to understand and model complex systems. Researchers often construct these graphs from different perspectives, leading to significant variations for the same problem. Despite its importance, research on causal graph comparison remains scarce.
  arXiv  Detail & Related papers  (2025-03-14T03:29:26Z)
• Causal Graphs Meet Thoughts: Enhancing Complex Reasoning in Graph-Augmented LLMs [4.701165676405066]
  It is critical not only to retrieve relevant information but also to provide causal reasoning and explainability. This paper proposes a novel pipeline that filters large knowledge graphs to emphasize cause-effect edges. Experiments on medical question-answering tasks show consistent gains, with up to a 10% absolute improvement.
  arXiv  Detail & Related papers  (2025-01-24T19:31:06Z)
• Reasoning with Graphs: Structuring Implicit Knowledge to Enhance LLMs Reasoning [73.2950349728376]
  Large language models (LLMs) have demonstrated remarkable success across a wide range of tasks. However, they still encounter challenges in reasoning tasks that require understanding and inferring relationships between pieces of information. This challenge is particularly pronounced in tasks involving multi-step processes, such as logical reasoning and multi-hop question answering. We propose Reasoning with Graphs (RwG) by first constructing explicit graphs from the context.
  arXiv  Detail & Related papers  (2025-01-14T05:18:20Z)
• A Top-down Graph-based Tool for Modeling Classical Semantic Maps: A Crosslinguistic Case Study of Supplementary Adverbs [50.982315553104975]
  Semantic map models (SMMs) construct a network-like conceptual space from cross-linguistic instances or forms. Most SMMs are manually built by human experts using bottom-up procedures. We propose a novel graph-based algorithm that automatically generates conceptual spaces and SMMs in a top-down manner.
  arXiv  Detail & Related papers  (2024-12-02T12:06:41Z)
• Instance-Aware Graph Prompt Learning [71.26108600288308]
  We introduce Instance-Aware Graph Prompt Learning (IA-GPL) in this paper. The process involves generating intermediate prompts for each instance using a lightweight architecture. Experiments conducted on multiple datasets and settings showcase the superior performance of IA-GPL compared to state-of-the-art baselines.
  arXiv  Detail & Related papers  (2024-11-26T18:38:38Z)
• Towards Graph Foundation Models: The Perspective of Zero-shot Reasoning on Knowledge Graphs [14.392577069212292]
  We introduce SCORE, a unified graph reasoning framework that effectively generalizes diverse graph tasks using zero-shot learning. We evaluate SCORE using 38 diverse graph datasets, covering node-level, link-level, and graph-level tasks across multiple domains.
  arXiv  Detail & Related papers  (2024-10-16T14:26:08Z)
• Reasoning Graph Enhanced Exemplars Retrieval for In-Context Learning [13.381974811214764]
  Reasoning Graph-enhanced Exemplar Retrieval(RGER) RGER uses graph kernel to select exemplars with semantic and structural similarity. The efficacy of RGER on math and logit reasoning tasks showcases its superiority over state-of-the-art retrieval-based approaches.
  arXiv  Detail & Related papers  (2024-09-17T12:58:29Z)
• Can Graph Learning Improve Planning in LLM-based Agents? [61.47027387839096]
  Task planning in language agents is emerging as an important research topic alongside the development of large language models (LLMs) In this paper, we explore graph learning-based methods for task planning, a direction that is to the prevalent focus on prompt design. Our interest in graph learning stems from a theoretical discovery: the biases of attention and auto-regressive loss impede LLMs' ability to effectively navigate decision-making on graphs.
  arXiv  Detail & Related papers  (2024-05-29T14:26:24Z)
• Improving Complex Reasoning over Knowledge Graph with Logic-Aware Curriculum Tuning [89.89857766491475]
  We propose a complex reasoning schema over KG upon large language models (LLMs) We augment the arbitrary first-order logical queries via binary tree decomposition to stimulate the reasoning capability of LLMs. Experiments across widely used datasets demonstrate that LACT has substantial improvements(brings an average +5.5% MRR score) over advanced methods.
  arXiv  Detail & Related papers  (2024-05-02T18:12:08Z)
• ULTRA-DP: Unifying Graph Pre-training with Multi-task Graph Dual Prompt [67.8934749027315]
  We propose a unified framework for graph hybrid pre-training which injects the task identification and position identification into GNNs. We also propose a novel pre-training paradigm based on a group of $k$-nearest neighbors.
  arXiv  Detail & Related papers  (2023-10-23T12:11:13Z)
• Beyond Chain-of-Thought, Effective Graph-of-Thought Reasoning in Language Models [74.40196814292426]
  We propose Graph-of-Thought (GoT) reasoning, which models human thought processes not only as a chain but also as a graph. GoT captures the non-sequential nature of human thinking and allows for a more realistic modeling of thought processes. We evaluate GoT's performance on a text-only reasoning task and a multimodal reasoning task.
  arXiv  Detail & Related papers  (2023-05-26T02:15:09Z)
• Conversational Semantic Parsing using Dynamic Context Graphs [68.72121830563906]
  We consider the task of conversational semantic parsing over general purpose knowledge graphs (KGs) with millions of entities, and thousands of relation-types. We focus on models which are capable of interactively mapping user utterances into executable logical forms.
  arXiv  Detail & Related papers  (2023-05-04T16:04:41Z)
• FactGraph: Evaluating Factuality in Summarization with Semantic Graph Representations [114.94628499698096]
  We propose FactGraph, a method that decomposes the document and the summary into structured meaning representations (MRs) MRs describe core semantic concepts and their relations, aggregating the main content in both document and summary in a canonical form, and reducing data sparsity. Experiments on different benchmarks for evaluating factuality show that FactGraph outperforms previous approaches by up to 15%.
  arXiv  Detail & Related papers  (2022-04-13T16:45:33Z)
• Graph Collaborative Reasoning [18.45161138837384]
  Graph Collaborative Reasoning (GCR) can use the neighbor link information for relational reasoning on graphs from logical reasoning perspectives. We provide a simple approach to translate a graph structure into logical expressions, so that the link prediction task can be converted into a neural logic reasoning problem. To show the effectiveness of our work, we conduct experiments on graph-related tasks such as link prediction and recommendation based on commonly used benchmark datasets.
  arXiv  Detail & Related papers  (2021-12-27T14:27:58Z)
• Joint Graph Learning and Matching for Semantic Feature Correspondence [69.71998282148762]
  We propose a joint emphgraph learning and matching network, named GLAM, to explore reliable graph structures for boosting graph matching. The proposed method is evaluated on three popular visual matching benchmarks (Pascal VOC, Willow Object and SPair-71k) It outperforms previous state-of-the-art graph matching methods by significant margins on all benchmarks.
  arXiv  Detail & Related papers  (2021-09-01T08:24:02Z)
• Learning the Implicit Semantic Representation on Graph-Structured Data [57.670106959061634]
  Existing representation learning methods in graph convolutional networks are mainly designed by describing the neighborhood of each node as a perceptual whole. We propose a Semantic Graph Convolutional Networks (SGCN) that explores the implicit semantics by learning latent semantic-paths in graphs.
  arXiv  Detail & Related papers  (2021-01-16T16:18:43Z)
• MrGCN: Mirror Graph Convolution Network for Relation Extraction with Long-Term Dependencies [32.27755470353054]
  In relation extraction, dependency trees that contain rich syntactic clues have been widely used to help capture long-term dependencies in text. We propose the Mirror Graph Convolution Network (MrGCN), a GNN model with pooling-unpooling structures tailored to relation extraction. Experiments on two datasets demonstrate the effectiveness of our method, showing significant improvements over previous results.
  arXiv  Detail & Related papers  (2021-01-01T00:52:53Z)
• Out-of-Sample Representation Learning for Multi-Relational Graphs [8.956321788625894]
  We study the out-of-sample representation learning problem for non-attributed knowledge graphs. We create benchmark datasets for this task, develop several models and baselines, and provide empirical analyses and comparisons of the proposed models and baselines.
  arXiv  Detail & Related papers  (2020-04-28T00:53:01Z)
• Evaluating Logical Generalization in Graph Neural Networks [59.70452462833374]
  We study the task of logical generalization using graph neural networks (GNNs) Our benchmark suite, GraphLog, requires that learning algorithms perform rule induction in different synthetic logics. We find that the ability for models to generalize and adapt is strongly determined by the diversity of the logical rules they encounter during training.
  arXiv  Detail & Related papers  (2020-03-14T05:45:55Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.