CADGE: Context-Aware Dialogue Generation Enhanced with Graph-Structured
Knowledge Aggregation
- URL: http://arxiv.org/abs/2305.06294v2
- Date: Wed, 7 Jun 2023 05:33:07 GMT
- Title: CADGE: Context-Aware Dialogue Generation Enhanced with Graph-Structured
Knowledge Aggregation
- Authors: Hongbo Zhang, Chen Tang, Tyler Loakman, Chenghua Lin and Stefan Goetze
- Abstract summary: Commonsense knowledge is crucial to many natural language processing tasks.
Existing works usually incorporate graph knowledge with conventional graph neural networks (GNNs)
We argue that these separate representation learning stages may be suboptimal for neural networks to learn the overall context contained in both types of input knowledge.
- Score: 21.331251731675668
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Commonsense knowledge is crucial to many natural language processing tasks.
Existing works usually incorporate graph knowledge with conventional graph
neural networks (GNNs), leading to the text and graph knowledge encoding
processes being separated in a serial pipeline. We argue that these separate
representation learning stages may be suboptimal for neural networks to learn
the overall context contained in both types of input knowledge. In this paper,
we propose a novel context-aware graph-attention model (Context-aware GAT),
which can effectively incorporate global features of relevant knowledge graphs
based on a context-enhanced knowledge aggregation process. Specifically, our
framework leverages a novel representation learning approach to process
heterogeneous features - combining flattened graph knowledge with text. To the
best of our knowledge, this is the first attempt at hierarchically applying
graph knowledge aggregation on a connected subgraph in addition to contextual
information to support commonsense dialogue generation. This framework shows
superior performance compared to conventional GNN-based language frameworks.
Both automatic and human evaluation demonstrates that our proposed model has
significant performance uplifts over state-of-the-art baselines.
Related papers
- Bridging Local Details and Global Context in Text-Attributed Graphs [62.522550655068336]
GraphBridge is a framework that bridges local and global perspectives by leveraging contextual textual information.
Our method achieves state-of-theart performance, while our graph-aware token reduction module significantly enhances efficiency and solves scalability issues.
arXiv Detail & Related papers (2024-06-18T13:35:25Z) - Enhancing Dialogue Generation via Dynamic Graph Knowledge Aggregation [23.54754465832362]
In conventional graph neural networks (GNNs) message passing on a graph is independent from text.
This training regime leads to a semantic gap between graph knowledge and text.
We propose a novel framework for knowledge graph enhanced dialogue generation.
arXiv Detail & Related papers (2023-06-28T13:21:00Z) - State of the Art and Potentialities of Graph-level Learning [54.68482109186052]
Graph-level learning has been applied to many tasks including comparison, regression, classification, and more.
Traditional approaches to learning a set of graphs rely on hand-crafted features, such as substructures.
Deep learning has helped graph-level learning adapt to the growing scale of graphs by extracting features automatically and encoding graphs into low-dimensional representations.
arXiv Detail & Related papers (2023-01-14T09:15:49Z) - Knowledge Graph Augmented Network Towards Multiview Representation
Learning for Aspect-based Sentiment Analysis [96.53859361560505]
We propose a knowledge graph augmented network (KGAN) to incorporate external knowledge with explicitly syntactic and contextual information.
KGAN captures the sentiment feature representations from multiple perspectives, i.e., context-, syntax- and knowledge-based.
Experiments on three popular ABSA benchmarks demonstrate the effectiveness and robustness of our KGAN.
arXiv Detail & Related papers (2022-01-13T08:25:53Z) - KELM: Knowledge Enhanced Pre-Trained Language Representations with
Message Passing on Hierarchical Relational Graphs [26.557447199727758]
We propose a novel knowledge-aware language model framework based on fine-tuning process.
Our model can efficiently incorporate world knowledge from KGs into existing language models such as BERT.
arXiv Detail & Related papers (2021-09-09T12:39:17Z) - GraphFormers: GNN-nested Transformers for Representation Learning on
Textual Graph [53.70520466556453]
We propose GraphFormers, where layerwise GNN components are nested alongside the transformer blocks of language models.
With the proposed architecture, the text encoding and the graph aggregation are fused into an iterative workflow.
In addition, a progressive learning strategy is introduced, where the model is successively trained on manipulated data and original data to reinforce its capability of integrating information on graph.
arXiv Detail & Related papers (2021-05-06T12:20:41Z) - Edge: Enriching Knowledge Graph Embeddings with External Text [32.01476220906261]
We propose a knowledge graph enrichment and embedding framework named Edge.
Given an original knowledge graph, we first generate a rich but noisy augmented graph using external texts in semantic and structural level.
To distill the relevant knowledge and suppress the introduced noise, we design a graph alignment term in a shared embedding space between the original graph and augmented graph.
arXiv Detail & Related papers (2021-04-11T03:47:06Z) - Exploiting Structured Knowledge in Text via Graph-Guided Representation
Learning [73.0598186896953]
We present two self-supervised tasks learning over raw text with the guidance from knowledge graphs.
Building upon entity-level masked language models, our first contribution is an entity masking scheme.
In contrast to existing paradigms, our approach uses knowledge graphs implicitly, only during pre-training.
arXiv Detail & Related papers (2020-04-29T14:22:42Z) - A Heterogeneous Graph with Factual, Temporal and Logical Knowledge for
Question Answering Over Dynamic Contexts [81.4757750425247]
We study question answering over a dynamic textual environment.
We develop a graph neural network over the constructed graph, and train the model in an end-to-end manner.
arXiv Detail & Related papers (2020-04-25T04:53:54Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.