GraphIF: Enhancing Multi-Turn Instruction Following for Large Language Models with Relation Graph Prompt

• URL: http://arxiv.org/abs/2511.10051v1
• Date: Fri, 14 Nov 2025 01:28:56 GMT
• Title: GraphIF: Enhancing Multi-Turn Instruction Following for Large Language Models with Relation Graph Prompt
• Abstract summary: Multi-turn instruction following is essential for building intelligent conversational systems that adhere to instructions across dialogue turns.<n>Existing approaches to enhancing multi-turn instruction following primarily rely on collecting or generating large-scale multi-turn dialogue datasets.<n>We propose GraphIF, a plug-and-play framework that models multi-turn dialogues as directed relation graphs and leverages graph prompts to enhance instruction following capabilities.
• Score: 8.491557907744523
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Multi-turn instruction following is essential for building intelligent conversational systems that can consistently adhere to instructions across dialogue turns. However, existing approaches to enhancing multi-turn instruction following primarily rely on collecting or generating large-scale multi-turn dialogue datasets to fine-tune large language models (LLMs), which treat each response generation as an isolated task and fail to explicitly incorporate multi-turn instruction following into the optimization objectives. As a result, instruction-tuned LLMs often struggle with complex long-distance constraints. In multi-turn dialogues, relational constraints across turns can be naturally modeled as labeled directed edges, making graph structures particularly suitable for modeling multi-turn instruction following. Despite this potential, leveraging graph structures to enhance the multi-turn instruction following capabilities of LLMs remains unexplored. To bridge this gap, we propose GraphIF, a plug-and-play framework that models multi-turn dialogues as directed relation graphs and leverages graph prompts to enhance the instruction following capabilities of LLMs. GraphIF comprises three key components: (1) an agent-based relation extraction module that captures inter-turn semantic relations via action-triggered mechanisms to construct structured graphs; (2) a relation graph prompt generation module that converts structured graph information into natural language prompts; and (3) a response rewriting module that refines initial LLM outputs using the generated graph prompts. Extensive experiments on two long multi-turn dialogue datasets demonstrate that GraphIF can be seamlessly integrated into instruction-tuned LLMs and leads to significant improvements across all four multi-turn instruction-following evaluation metrics.

Related papers

• MSRAMIE: Multimodal Structured Reasoning Agent for Multi-instruction Image Editing [6.097400596282323]
  We propose MSRAMIE, a training-free agent framework built on Multimodal Large Language Model (MLLM)<n>MSRAMIE takes existing editing models as plug-in components and handle multi-instruction tasks via structured multimodal reasoning.<n> Experiments show that as the instruction complexity increases, MSRAMIE can improve instruction following over 15% and increases the probability of finishing all modifications in a single run over 100%.
  arXiv  Detail & Related papers  (2026-03-17T09:05:16Z)
• Graph4MM: Weaving Multimodal Learning with Structural Information [52.16646463590474]
  Graphs provide powerful structural information for modeling intra- and inter-modal relationships.<n>Previous works fail to distinguish multi-hop neighbors and treat the graph as a standalone modality.<n>We propose Graph4MM, a graph-based multimodal learning framework.
  arXiv  Detail & Related papers  (2025-10-19T20:13:03Z)
• GRIL: Knowledge Graph Retrieval-Integrated Learning with Large Language Models [59.72897499248909]
  We propose a novel graph retriever trained end-to-end with Large Language Models (LLMs)<n>Within the extracted subgraph, structural knowledge and semantic features are encoded via soft tokens and the verbalized graph, respectively, which are infused into the LLM together.<n>Our approach consistently achieves state-of-the-art performance, validating the strength of joint graph-LLM optimization for complex reasoning tasks.
  arXiv  Detail & Related papers  (2025-09-20T02:38:00Z)
• Quantizing Text-attributed Graphs for Semantic-Structural Integration [6.721504414917793]
  Text-attributed graphs (TAGs) have emerged as a powerful representation for modeling complex relationships across diverse domains.<n>With the rise of large language models (LLMs), there is growing interest in leveraging their capabilities for graph learning.<n>We propose STAG, a novel self-supervised framework that directly quantizes graph structural information into discrete tokens using a frozen codebook.
  arXiv  Detail & Related papers  (2025-07-20T09:18:02Z)
• MLaGA: Multimodal Large Language and Graph Assistant [9.985787670804823]
  Large Language Models (LLMs) have demonstrated substantial efficacy in advancing graph-structured data analysis.<n>We introduce the Multimodal Large Language and Graph Assistant (MLaGA), an innovative model that adeptly extends LLM capabilities to facilitate reasoning over complex graph structures and multimodal attributes.
  arXiv  Detail & Related papers  (2025-06-03T07:52:00Z)
• LLM as GNN: Graph Vocabulary Learning for Text-Attributed Graph Foundation Models [87.68057302738457]
  Text-Attributed Graphs (TAGs) are ubiquitous in real-world scenarios.<n>Despite large efforts to integrate Large Language Models (LLMs) and Graph Neural Networks (GNNs) for TAGs, existing approaches suffer from decoupled architectures.<n>We propose PromptGFM, a versatile GFM for TAGs grounded in graph vocabulary learning.
  arXiv  Detail & Related papers  (2025-03-05T09:45:22Z)
• Graph-oriented Instruction Tuning of Large Language Models for Generic Graph Mining [36.366468326267004]
  MuseGraph is a framework for graph mining across tasks and datasets.<n>It seamlessly integrates the strengths of Graph Neural Networks (GNNs) and Large Language Models (LLMs)<n>Our experimental results demonstrate significant improvements in five graph tasks and ten datasets.
  arXiv  Detail & Related papers  (2024-03-02T09:27:32Z)
• Structure Guided Prompt: Instructing Large Language Model in Multi-Step Reasoning by Exploring Graph Structure of the Text [44.81698187939784]
  This paper introduces Structure Guided Prompt, a framework designed to improve the multi-step reasoning capabilities of Large Language Models (LLMs) Our experiments show that this framework significantly enhances the reasoning capabilities of LLMs, enabling them to excel in a broader spectrum of natural language scenarios.
  arXiv  Detail & Related papers  (2024-02-20T22:56:23Z)
• Disentangled Representation Learning with Large Language Models for Text-Attributed Graphs [57.052160123387104]
  We present the Disentangled Graph-Text Learner (DGTL) model, which is able to enhance the reasoning and predicting capabilities of LLMs for TAGs. Our proposed DGTL model incorporates graph structure information through tailored disentangled graph neural network (GNN) layers. Experimental evaluations demonstrate the effectiveness of the proposed DGTL model on achieving superior or comparable performance over state-of-the-art baselines.
  arXiv  Detail & Related papers  (2023-10-27T14:00:04Z)
• Multimodal Graph Learning for Generative Tasks [89.44810441463652]
  Multimodal learning combines multiple data modalities, broadening the types and complexity of data our models can utilize. We propose Multimodal Graph Learning (MMGL), a framework for capturing information from multiple multimodal neighbors with relational structures among them.
  arXiv  Detail & Related papers  (2023-10-11T13:25:03Z)
• Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructions [126.3136109870403]
  We introduce a generic and lightweight Visual Prompt Generator Complete module (VPG-C) VPG-C infers and completes the missing details essential for comprehending demonstrative instructions. We build DEMON, a comprehensive benchmark for demonstrative instruction understanding.
  arXiv  Detail & Related papers  (2023-08-08T09:32:43Z)
• Graphologue: Exploring Large Language Model Responses with Interactive Diagrams [6.79341019029299]
  Large language models (LLMs) have recently soared in popularity due to their ease of access and the unprecedented ability to synthesize text responses to diverse user questions. We present Graphologue, an interactive system that converts text-based responses from LLMs into graphical diagrams to facilitate information-seeking and question-answering tasks.
  arXiv  Detail & Related papers  (2023-05-19T06:53:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.