Knowledge Graph Enhanced Large Language Model Editing
- URL: http://arxiv.org/abs/2402.13593v1
- Date: Wed, 21 Feb 2024 07:52:26 GMT
- Title: Knowledge Graph Enhanced Large Language Model Editing
- Authors: Mengqi Zhang, Xiaotian Ye, Qiang Liu, Pengjie Ren, Shu Wu, Zhumin Chen
- Abstract summary: Large language models (LLMs) are pivotal in advancing natural language processing (NLP) tasks.
Existing editing methods struggle to track and incorporate changes in knowledge associated with edits.
We propose a novel model editing method that leverages knowledge graphs for enhancing LLM editing, namely GLAME.
- Score: 37.6721061644483
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Large language models (LLMs) are pivotal in advancing natural language
processing (NLP) tasks, yet their efficacy is hampered by inaccuracies and
outdated knowledge. Model editing emerges as a promising solution to address
these challenges. However, existing editing methods struggle to track and
incorporate changes in knowledge associated with edits, which limits the
generalization ability of postedit LLMs in processing edited knowledge. To
tackle these problems, we propose a novel model editing method that leverages
knowledge graphs for enhancing LLM editing, namely GLAME. Specifically, we
first utilize a knowledge graph augmentation module to uncover associated
knowledge that has changed due to editing, obtaining its internal
representations within LLMs. This approach allows knowledge alterations within
LLMs to be reflected through an external graph structure. Subsequently, we
design a graph-based knowledge edit module to integrate structured knowledge
into the model editing. This ensures that the updated parameters reflect not
only the modifications of the edited knowledge but also the changes in other
associated knowledge resulting from the editing process. Comprehensive
experiments conducted on GPT-J and GPT-2 XL demonstrate that GLAME
significantly improves the generalization capabilities of post-edit LLMs in
employing edited knowledge.
Related papers
- Uncovering Overfitting in Large Language Model Editing [35.55260822503773]
We identify and investigate the phenomenon of Editing Overfit, where edited models assign disproportionately high probabilities to the edit target.
We propose a new plug-and-play strategy called Learn to Inference (LTI), which introduce a Multi-stage Inference Constraint module to guide the edited models in recalling new knowledge.
arXiv Detail & Related papers (2024-10-10T11:09:00Z) - AlphaEdit: Null-Space Constrained Knowledge Editing for Language Models [65.93240009586351]
Large language models (LLMs) often exhibit hallucinations due to incorrect or outdated knowledge.
We introduce AlphaEdit, a novel solution that projects perturbation onto the null space of the preserved knowledge before applying it to the parameters.
We theoretically prove that this projection ensures the output of post-edited LLMs remains unchanged when queried about the preserved knowledge.
arXiv Detail & Related papers (2024-10-03T10:06:27Z) - Editing the Mind of Giants: An In-Depth Exploration of Pitfalls of Knowledge Editing in Large Language Models [26.516571783335824]
Recent studies have identified side effects, such as knowledge distortion and the deterioration of general abilities, that have emerged after editing.
This survey presents a comprehensive study of these side effects, providing a unified perspective on the challenges of knowledge editing in large language models.
arXiv Detail & Related papers (2024-06-03T15:28:21Z) - Editing Conceptual Knowledge for Large Language Models [65.38231526537476]
This paper pioneers the investigation of editing conceptual knowledge for Large Language Models (LLMs)
We construct a novel benchmark dataset ConceptEdit and establish a suite of new metrics for evaluation.
experimental results reveal that, although existing editing methods can efficiently modify concept-level definition to some extent, they also have the potential to distort the related instantial knowledge.
arXiv Detail & Related papers (2024-03-10T16:57:10Z) - Learning to Edit: Aligning LLMs with Knowledge Editing [101.96620267293731]
We propose a Learning to Edit (LTE) framework, focusing on teaching large language models to apply updated knowledge into input questions.
LTE features a two-phase process: (i) the Alignment Phase, which fine-tunes LLMs on a meticulously curated parallel dataset to make reliable, in-scope edits.
We demonstrate LTE's superiority in knowledge editing performance, robustness in both batch and sequential editing, minimal interference on general tasks, and rapid editing speeds.
arXiv Detail & Related papers (2024-02-19T07:45:17Z) - WilKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editing [19.357663224043534]
This study reveals a performance degradation encountered by knowledge editing in lifelong editing.
We introduce a knowledge editing approach named Wise-Layer Knowledge Editor (WilKE)
WilKE selects editing layer based on the pattern matching degree of editing knowledge across different layers in language models.
arXiv Detail & Related papers (2024-02-16T05:29:59Z) - Model Editing Harms General Abilities of Large Language Models: Regularization to the Rescue [122.20016030723043]
We evaluate the side effects of model editing on large language models (LLMs)
Our analysis reveals that the side effects are caused by model editing altering the original model weights excessively.
To mitigate this, a method named RECT is proposed to regularize the edit update weights.
arXiv Detail & Related papers (2024-01-09T18:03:15Z) - A Comprehensive Study of Knowledge Editing for Large Language Models [82.65729336401027]
Large Language Models (LLMs) have shown extraordinary capabilities in understanding and generating text that closely mirrors human communication.
This paper defines the knowledge editing problem and provides a comprehensive review of cutting-edge approaches.
We introduce a new benchmark, KnowEdit, for a comprehensive empirical evaluation of representative knowledge editing approaches.
arXiv Detail & Related papers (2024-01-02T16:54:58Z) - EasyEdit: An Easy-to-use Knowledge Editing Framework for Large Language Models [45.70959260613425]
We propose EasyEdit, an easy-to-use knowledge editing framework for Large Language Models.
It supports various cutting-edge knowledge editing approaches and can be readily applied to many well-known LLMs.
We report the knowledge editing results on LlaMA-2 with EasyEdit, demonstrating that knowledge editing surpasses traditional fine-tuning.
arXiv Detail & Related papers (2023-08-14T16:52:42Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.