Inferential Text Generation with Multiple Knowledge Sources and Meta-Learning

• URL: http://arxiv.org/abs/2004.03070v2
• Date: Wed, 15 Apr 2020 05:02:26 GMT
• Title: Inferential Text Generation with Multiple Knowledge Sources and Meta-Learning
• Authors: Daya Guo, Akari Asai, Duyu Tang, Nan Duan, Ming Gong, Linjun Shou, Daxin Jiang, Jian Yin and Ming Zhou
• Abstract summary: We study the problem of generating inferential texts of events for a variety of commonsense like textitif-else relations. Existing approaches typically use limited evidence from training examples and learn for each relation individually. In this work, we use multiple knowledge sources as fuels for the model.
• Score: 117.23425857240679
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We study the problem of generating inferential texts of events for a variety of commonsense like \textit{if-else} relations. Existing approaches typically use limited evidence from training examples and learn for each relation individually. In this work, we use multiple knowledge sources as fuels for the model. Existing commonsense knowledge bases like ConceptNet are dominated by taxonomic knowledge (e.g., \textit{isA} and \textit{relatedTo} relations), having a limited number of inferential knowledge. We use not only structured commonsense knowledge bases, but also natural language snippets from search-engine results. These sources are incorporated into a generative base model via key-value memory network. In addition, we introduce a meta-learning based multi-task learning algorithm. For each targeted commonsense relation, we regard the learning of examples from other relations as the meta-training process, and the evaluation on examples from the targeted relation as the meta-test process. We conduct experiments on Event2Mind and ATOMIC datasets. Results show that both the integration of multiple knowledge sources and the use of the meta-learning algorithm improve the performance.

Related papers

• Generative Multi-Modal Knowledge Retrieval with Large Language Models [75.70313858231833]
  We propose an innovative end-to-end generative framework for multi-modal knowledge retrieval. Our framework takes advantage of the fact that large language models (LLMs) can effectively serve as virtual knowledge bases. We demonstrate significant improvements ranging from 3.0% to 14.6% across all evaluation metrics when compared to strong baselines.
  arXiv  Detail & Related papers  (2024-01-16T08:44:29Z)
• KETM:A Knowledge-Enhanced Text Matching method [0.0]
  We introduce a new model for text matching called the Knowledge Enhanced Text Matching model (KETM) We use Wiktionary to retrieve the text word definitions as our external knowledge. We fuse text and knowledge using a gating mechanism to learn the ratio of text and knowledge fusion.
  arXiv  Detail & Related papers  (2023-08-11T17:08:14Z)
• ASPER: Answer Set Programming Enhanced Neural Network Models for Joint Entity-Relation Extraction [11.049915720093242]
  This paper proposes a new approach, ASP-enhanced Entity-Relation extraction (ASPER) ASPER jointly recognizes entities and relations by learning from both data and domain knowledge. In particular, ASPER takes advantage of the factual knowledge (represented as facts in ASP) and derived knowledge (represented as rules in ASP) in the learning process of neural network models.
  arXiv  Detail & Related papers  (2023-05-24T17:32:58Z)
• Meta Knowledge Condensation for Federated Learning [65.20774786251683]
  Existing federated learning paradigms usually extensively exchange distributed models at a central solver to achieve a more powerful model. This would incur severe communication burden between a server and multiple clients especially when data distributions are heterogeneous. Unlike existing paradigms, we introduce an alternative perspective to significantly decrease the communication cost in federate learning.
  arXiv  Detail & Related papers  (2022-09-29T15:07:37Z)
• A Unified Continuous Learning Framework for Multi-modal Knowledge Discovery and Pre-training [73.7507857547549]
  We propose to unify knowledge discovery and multi-modal pre-training in a continuous learning framework. For knowledge discovery, a pre-trained model is used to identify cross-modal links on a graph. For model pre-training, the knowledge graph is used as the external knowledge to guide the model updating.
  arXiv  Detail & Related papers  (2022-06-11T16:05:06Z)
• Commonsense and Named Entity Aware Knowledge Grounded Dialogue Generation [20.283091595536835]
  We present a novel open-domain dialogue generation model which effectively utilizes the large-scale commonsense and named entity based knowledge. Our proposed model utilizes a multi-hop attention layer to preserve the most accurate and critical parts of the dialogue history and the associated knowledge. Empirical results on two benchmark dataset demonstrate that our model significantly outperforms the state-of-the-art methods in terms of both automatic evaluation metrics and human judgment.
  arXiv  Detail & Related papers  (2022-05-27T12:11:40Z)
• One-shot Scene Graph Generation [130.57405850346836]
  We propose Multiple Structured Knowledge (Relational Knowledgesense Knowledge) for the one-shot scene graph generation task. Our method significantly outperforms existing state-of-the-art methods by a large margin.
  arXiv  Detail & Related papers  (2022-02-22T11:32:59Z)
• Commonsense Knowledge in Word Associations and ConceptNet [37.751909219863585]
  This paper presents an in-depth comparison of two large-scale resources of general knowledge: ConcpetNet and SWOW. We examine the structure, overlap and differences between the two graphs, as well as the extent to which they encode situational commonsense knowledge.
  arXiv  Detail & Related papers  (2021-09-20T06:06:30Z)
• LEAPME: Learning-based Property Matching with Embeddings [5.2078071454435815]
  We present a new machine learning-based property matching approach called LEAPME (LEArning-based Property Matching with Embeddings) The approach heavily makes use of word embeddings to better utilize the domain-specific semantics of both property names and instance values. Our comparative evaluation against five baselines for several multi-source datasets with real-world data shows the high effectiveness of LEAPME.
  arXiv  Detail & Related papers  (2020-10-05T12:42:39Z)
• Language Generation with Multi-Hop Reasoning on Commonsense Knowledge Graph [124.45799297285083]
  We argue that exploiting both the structural and semantic information of the knowledge graph facilitates commonsense-aware text generation. We propose Generation with Multi-Hop Reasoning Flow (GRF) that enables pre-trained models with dynamic multi-hop reasoning on multi-relational paths extracted from the external commonsense knowledge graph.
  arXiv  Detail & Related papers  (2020-09-24T13:55:32Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.