LEKA:LLM-Enhanced Knowledge Augmentation

• URL: http://arxiv.org/abs/2501.17802v1
• Date: Wed, 29 Jan 2025 17:44:57 GMT
• Title: LEKA:LLM-Enhanced Knowledge Augmentation
• Authors: Xinhao Zhang, Jinghan Zhang, Fengran Mo, Dongjie Wang, Yanjie Fu, Kunpeng Liu,
• Abstract summary: Humans excel in analogical learning and knowledge transfer. Models would transition from passively acquiring to actively accessing and learning from knowledge. We develop a knowledge augmentation method LEKA for knowledge transfer.
• Score: 24.552995956148145
• License:
• Abstract: Humans excel in analogical learning and knowledge transfer and, more importantly, possess a unique understanding of identifying appropriate sources of knowledge. From a model's perspective, this presents an interesting challenge. If models could autonomously retrieve knowledge useful for transfer or decision-making to solve problems, they would transition from passively acquiring to actively accessing and learning from knowledge. However, filling models with knowledge is relatively straightforward -- it simply requires more training and accessible knowledge bases. The more complex task is teaching models about which knowledge can be analogized and transferred. Therefore, we design a knowledge augmentation method LEKA for knowledge transfer that actively searches for suitable knowledge sources that can enrich the target domain's knowledge. This LEKA method extracts key information from textual information from the target domain, retrieves pertinent data from external data libraries, and harmonizes retrieved data with the target domain data in feature space and marginal probability measures. We validate the effectiveness of our approach through extensive experiments across various domains and demonstrate significant improvements over traditional methods in reducing computational costs, automating data alignment, and optimizing transfer learning outcomes.

Related papers

• KBAlign: Efficient Self Adaptation on Specific Knowledge Bases [73.34893326181046]
  Large language models (LLMs) usually rely on retrieval-augmented generation to exploit knowledge materials in an instant manner. We propose KBAlign, an approach designed for efficient adaptation to downstream tasks involving knowledge bases. Our method utilizes iterative training with self-annotated data such as Q&A pairs and revision suggestions, enabling the model to grasp the knowledge content efficiently.
  arXiv  Detail & Related papers  (2024-11-22T08:21:03Z)
• Fine-Grained Stateful Knowledge Exploration: A Novel Paradigm for Integrating Knowledge Graphs with Large Language Models [19.049828741139425]
  Large Language Models (LLMs) have shown impressive capabilities, yet updating their knowledge remains a significant challenge. Most existing methods use a paradigm that treats the question as the objective, with relevant knowledge being incrementally retrieved from the knowledge graph. We propose a novel paradigm of fine-grained stateful knowledge exploration, which addresses the information granularity mismatch' issue.
  arXiv  Detail & Related papers  (2024-01-24T13:36:50Z)
• Bridged-GNN: Knowledge Bridge Learning for Effective Knowledge Transfer [65.42096702428347]
  Graph Neural Networks (GNNs) aggregate information from neighboring nodes. Knowledge Bridge Learning (KBL) learns a knowledge-enhanced posterior distribution for target domains. Bridged-GNN includes an Adaptive Knowledge Retrieval module to build Bridged-Graph and a Graph Knowledge Transfer module.
  arXiv  Detail & Related papers  (2023-08-18T12:14:51Z)
• Graph Enabled Cross-Domain Knowledge Transfer [1.52292571922932]
  Cross-Domain Knowledge Transfer is an approach to mitigate the gap between good representation learning and the scarce knowledge in the domain of interest. From the machine learning perspective, the paradigm of semi-supervised learning takes advantage of large amount of data without ground truth and achieves impressive learning performance improvement.
  arXiv  Detail & Related papers  (2023-04-07T03:02:10Z)
• Learning with Recoverable Forgetting [77.56338597012927]
  Learning wIth Recoverable Forgetting explicitly handles the task- or sample-specific knowledge removal and recovery. Specifically, LIRF brings in two innovative schemes, namely knowledge deposit and withdrawal. We conduct experiments on several datasets, and demonstrate that the proposed LIRF strategy yields encouraging results with gratifying generalization capability.
  arXiv  Detail & Related papers  (2022-07-17T16:42:31Z)
• Transferability in Deep Learning: A Survey [80.67296873915176]
  The ability to acquire and reuse knowledge is known as transferability in deep learning. We present this survey to connect different isolated areas in deep learning with their relation to transferability. We implement a benchmark and an open-source library, enabling a fair evaluation of deep learning methods in terms of transferability.
  arXiv  Detail & Related papers  (2022-01-15T15:03:17Z)
• Kformer: Knowledge Injection in Transformer Feed-Forward Layers [107.71576133833148]
  We propose a novel knowledge fusion model, namely Kformer, which incorporates external knowledge through the feed-forward layer in Transformer. We empirically find that simply injecting knowledge into FFN can facilitate the pre-trained language model's ability and facilitate current knowledge fusion methods.
  arXiv  Detail & Related papers  (2022-01-15T03:00:27Z)
• Data-Free Knowledge Transfer: A Survey [13.335198869928167]
  knowledge distillation (KD) and domain adaptation (DA) are proposed and become research highlights. They both aim to transfer useful information from a well-trained model with original training data. Recently, the data-free knowledge transfer paradigm has attracted appealing attention.
  arXiv  Detail & Related papers  (2021-12-31T03:39:42Z)
• Fractional Transfer Learning for Deep Model-Based Reinforcement Learning [0.966840768820136]
  Reinforcement learning (RL) is well known for requiring large amounts of data in order for RL agents to learn to perform complex tasks. Recent progress in model-based RL allows agents to be much more data-efficient. We present a simple alternative approach: fractional transfer learning.
  arXiv  Detail & Related papers  (2021-08-14T12:44:42Z)
• Towards a Universal Continuous Knowledge Base [49.95342223987143]
  We propose a method for building a continuous knowledge base that can store knowledge imported from multiple neural networks. Experiments on text classification show promising results. We import the knowledge from multiple models to the knowledge base, from which the fused knowledge is exported back to a single model.
  arXiv  Detail & Related papers  (2020-12-25T12:27:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.