Subgraph Retrieval Enhanced by Graph-Text Alignment for Commonsense Question Answering

• URL: http://arxiv.org/abs/2411.06866v1
• Date: Mon, 11 Nov 2024 10:57:31 GMT
• Title: Subgraph Retrieval Enhanced by Graph-Text Alignment for Commonsense Question Answering
• Authors: Boci Peng, Yongchao Liu, Xiaohe Bo, Sheng Tian, Baokun Wang, Chuntao Hong, Yan Zhang,
• Abstract summary: Commonsense question answering is a crucial task that requires machines to employ reasoning according to commonsense. Previous studies predominantly employ an extracting-and-modeling paradigm to harness the information in KG. We propose a novel framework: textbfSubgraph RtextbfEtrieval Enhanced by GratextbfPh-textbfText textbfAlignment, named textbfSEPTA.
• Score: 6.9841561321072465
• License:
• Abstract: Commonsense question answering is a crucial task that requires machines to employ reasoning according to commonsense. Previous studies predominantly employ an extracting-and-modeling paradigm to harness the information in KG, which first extracts relevant subgraphs based on pre-defined rules and then proceeds to design various strategies aiming to improve the representations and fusion of the extracted structural knowledge. Despite their effectiveness, there are still two challenges. On one hand, subgraphs extracted by rule-based methods may have the potential to overlook critical nodes and result in uncontrollable subgraph size. On the other hand, the misalignment between graph and text modalities undermines the effectiveness of knowledge fusion, ultimately impacting the task performance. To deal with the problems above, we propose a novel framework: \textbf{S}ubgraph R\textbf{E}trieval Enhanced by Gra\textbf{P}h-\textbf{T}ext \textbf{A}lignment, named \textbf{SEPTA}. Firstly, we transform the knowledge graph into a database of subgraph vectors and propose a BFS-style subgraph sampling strategy to avoid information loss, leveraging the analogy between BFS and the message-passing mechanism. In addition, we propose a bidirectional contrastive learning approach for graph-text alignment, which effectively enhances both subgraph retrieval and knowledge fusion. Finally, all the retrieved information is combined for reasoning in the prediction module. Extensive experiments on five datasets demonstrate the effectiveness and robustness of our framework.

Related papers

• Explanation Graph Generation via Generative Pre-training over Synthetic Graphs [6.25568933262682]
  The generation of explanation graphs is a significant task that aims to produce explanation graphs in response to user input. Current research commonly fine-tunes a text-based pre-trained language model on a small downstream dataset that is annotated with labeled graphs. We propose a novel pre-trained framework EG3P(for Explanation Graph Generation via Generative Pre-training over synthetic graphs) for the explanation graph generation task.
  arXiv  Detail & Related papers  (2023-06-01T13:20:22Z)
• Scientific Paper Extractive Summarization Enhanced by Citation Graphs [50.19266650000948]
  We focus on leveraging citation graphs to improve scientific paper extractive summarization under different settings. Preliminary results demonstrate that citation graph is helpful even in a simple unsupervised framework. Motivated by this, we propose a Graph-based Supervised Summarization model (GSS) to achieve more accurate results on the task when large-scale labeled data are available.
  arXiv  Detail & Related papers  (2022-12-08T11:53:12Z)
• Unsupervised Extractive Summarization with Heterogeneous Graph Embeddings for Chinese Document [5.9630342951482085]
  We propose an unsupervised extractive summarizaiton method with heterogeneous graph embeddings (HGEs) for Chinese document. Experimental results demonstrate that our method consistently outperforms the strong baseline in three summarization datasets.
  arXiv  Detail & Related papers  (2022-11-09T06:07:31Z)
• VEM$^2$L: A Plug-and-play Framework for Fusing Text and Structure Knowledge on Sparse Knowledge Graph Completion [14.537509860565706]
  We propose a plug-and-play framework VEM2L over sparse Knowledge Graphs to fuse knowledge extracted from text and structure messages into a unity. Specifically, we partition knowledge acquired by models into two nonoverlapping parts. We also propose a new fusion strategy proved by Variational EM algorithm to fuse the generalization ability of models.
  arXiv  Detail & Related papers  (2022-07-04T15:50:21Z)
• Joint Graph Learning and Matching for Semantic Feature Correspondence [69.71998282148762]
  We propose a joint emphgraph learning and matching network, named GLAM, to explore reliable graph structures for boosting graph matching. The proposed method is evaluated on three popular visual matching benchmarks (Pascal VOC, Willow Object and SPair-71k) It outperforms previous state-of-the-art graph matching methods by significant margins on all benchmarks.
  arXiv  Detail & Related papers  (2021-09-01T08:24:02Z)
• BASS: Boosting Abstractive Summarization with Unified Semantic Graph [49.48925904426591]
  BASS is a framework for Boosting Abstractive Summarization based on a unified Semantic graph. A graph-based encoder-decoder model is proposed to improve both the document representation and summary generation process. Empirical results show that the proposed architecture brings substantial improvements for both long-document and multi-document summarization tasks.
  arXiv  Detail & Related papers  (2021-05-25T16:20:48Z)
• Recognizing Predictive Substructures with Subgraph Information Bottleneck [97.19131149357234]
  We propose a novel subgraph information bottleneck (SIB) framework to recognize such subgraphs, named IB-subgraph. Intractability of mutual information and the discrete nature of graph data makes the objective of SIB notoriously hard to optimize. Experiments on graph learning and large-scale point cloud tasks demonstrate the superior property of IB-subgraph.
  arXiv  Detail & Related papers  (2021-03-20T11:19:43Z)
• Graph Information Bottleneck for Subgraph Recognition [103.37499715761784]
  We propose a framework of Graph Information Bottleneck (GIB) for the subgraph recognition problem in deep graph learning. Under this framework, one can recognize the maximally informative yet compressive subgraph, named IB-subgraph. We evaluate the properties of the IB-subgraph in three application scenarios: improvement of graph classification, graph interpretation and graph denoising.
  arXiv  Detail & Related papers  (2020-10-12T09:32:20Z)
• Exploiting Structured Knowledge in Text via Graph-Guided Representation Learning [73.0598186896953]
  We present two self-supervised tasks learning over raw text with the guidance from knowledge graphs. Building upon entity-level masked language models, our first contribution is an entity masking scheme. In contrast to existing paradigms, our approach uses knowledge graphs implicitly, only during pre-training.
  arXiv  Detail & Related papers  (2020-04-29T14:22:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.