Modularized Transfer Learning with Multiple Knowledge Graphs for Zero-shot Commonsense Reasoning

• URL: http://arxiv.org/abs/2206.03715v1
• Date: Wed, 8 Jun 2022 07:36:31 GMT
• Title: Modularized Transfer Learning with Multiple Knowledge Graphs for Zero-shot Commonsense Reasoning
• Authors: Yu Jin Kim, Beong-woo Kwak, Youngwook Kim, Reinald Kim Amplayo, Seung-won Hwang, Jinyoung Yeo
• Abstract summary: A zero-shot QA system transforms a commonsense knowledge graph (KG) into synthetic QA-form samples for model training. Considering the increasing type of different commonsense KGs, this paper aims to extend the zero-shot transfer learning scenario into multiple-source settings. We propose to mitigate the loss of knowledge from the interference among the different knowledge sources, by developing a modular variant of the knowledge aggregation.
• Score: 22.443211209959497
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Commonsense reasoning systems should be able to generalize to diverse reasoning cases. However, most state-of-the-art approaches depend on expensive data annotations and overfit to a specific benchmark without learning how to perform general semantic reasoning. To overcome these drawbacks, zero-shot QA systems have shown promise as a robust learning scheme by transforming a commonsense knowledge graph (KG) into synthetic QA-form samples for model training. Considering the increasing type of different commonsense KGs, this paper aims to extend the zero-shot transfer learning scenario into multiple-source settings, where different KGs can be utilized synergetically. Towards this goal, we propose to mitigate the loss of knowledge from the interference among the different knowledge sources, by developing a modular variant of the knowledge aggregation as a new zero-shot commonsense reasoning framework. Results on five commonsense reasoning benchmarks demonstrate the efficacy of our framework, improving the performance with multiple KGs.

Related papers

• A Prompt-Based Knowledge Graph Foundation Model for Universal In-Context Reasoning [17.676185326247946]
  We propose a prompt-based KG foundation model via in-context learning, namely KG-ICL, to achieve a universal reasoning ability. To encode prompt graphs with the generalization ability to unseen entities and relations in queries, we first propose a unified tokenizer. Then, we propose two message passing neural networks to perform prompt encoding and KG reasoning, respectively.
  arXiv  Detail & Related papers  (2024-10-16T06:47:18Z)
• Common Knowledge Learning for Generating Transferable Adversarial Examples [60.1287733223249]
  This paper focuses on an important type of black-box attacks, where the adversary generates adversarial examples by a substitute (source) model. Existing methods tend to give unsatisfactory adversarial transferability when the source and target models are from different types of DNN architectures. We propose a common knowledge learning (CKL) framework to learn better network weights to generate adversarial examples.
  arXiv  Detail & Related papers  (2023-07-01T09:07:12Z)
• Recognizing Unseen Objects via Multimodal Intensive Knowledge Graph Propagation [68.13453771001522]
  We propose a multimodal intensive ZSL framework that matches regions of images with corresponding semantic embeddings. We conduct extensive experiments and evaluate our model on large-scale real-world data.
  arXiv  Detail & Related papers  (2023-06-14T13:07:48Z)
• Multi-hop Commonsense Knowledge Injection Framework for Zero-Shot Commonsense Question Answering [6.086719709100659]
  We propose a novel multi-hop commonsense knowledge injection framework. Our framework achieves state-of-art performance on five commonsense question answering benchmarks.
  arXiv  Detail & Related papers  (2023-05-10T07:13:47Z)
• From Mimicking to Integrating: Knowledge Integration for Pre-Trained Language Models [55.137869702763375]
  This paper explores a novel PLM reuse paradigm, Knowledge Integration (KI) KI aims to merge the knowledge from different teacher-PLMs, each of which specializes in a different classification problem, into a versatile student model. We then design a Model Uncertainty--aware Knowledge Integration (MUKI) framework to recover the golden supervision for the student.
  arXiv  Detail & Related papers  (2022-10-11T07:59:08Z)
• Great Truths are Always Simple: A Rather Simple Knowledge Encoder for Enhancing the Commonsense Reasoning Capacity of Pre-Trained Models [89.98762327725112]
  Commonsense reasoning in natural language is a desired ability of artificial intelligent systems. For solving complex commonsense reasoning tasks, a typical solution is to enhance pre-trained language models(PTMs) with a knowledge-aware graph neural network(GNN) encoder. Despite the effectiveness, these approaches are built on heavy architectures, and can't clearly explain how external knowledge resources improve the reasoning capacity of PTMs.
  arXiv  Detail & Related papers  (2022-05-04T01:27:36Z)
• elBERto: Self-supervised Commonsense Learning for Question Answering [131.51059870970616]
  We propose a Self-supervised Bidirectional Representation Learning of Commonsense framework, which is compatible with off-the-shelf QA model architectures. The framework comprises five self-supervised tasks to force the model to fully exploit the additional training signals from contexts containing rich commonsense. elBERto achieves substantial improvements on out-of-paragraph and no-effect questions where simple lexical similarity comparison does not help.
  arXiv  Detail & Related papers  (2022-03-17T16:23:45Z)
• Towards Robust Knowledge Graph Embedding via Multi-task Reinforcement Learning [44.38215560989223]
  Most existing knowledge graph embedding methods assume that all the triple facts in KGs are correct. This will lead to low-quality and unreliable representations of KGs. We propose a general multi-task reinforcement learning framework, which can greatly alleviate the noisy data problem.
  arXiv  Detail & Related papers  (2021-11-11T08:51:37Z)
• All About Knowledge Graphs for Actions [82.39684757372075]
  We propose a better understanding of knowledge graphs (KGs) that can be utilized for zero-shot and few-shot action recognition. We study three different construction mechanisms for KGs: action embeddings, action-object embeddings, visual embeddings. We present extensive analysis of the impact of different KGs on different experimental setups.
  arXiv  Detail & Related papers  (2020-08-28T01:44:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.