Alleviating the Knowledge-Language Inconsistency: A Study for Deep Commonsense Knowledge

• URL: http://arxiv.org/abs/2105.13607v2
• Date: Mon, 31 May 2021 13:09:19 GMT
• Title: Alleviating the Knowledge-Language Inconsistency: A Study for Deep Commonsense Knowledge
• Authors: Yi Zhang, Lei Li, Yunfang Wu, Qi Su, Xu Sun
• Abstract summary: Deep commonsense knowledge occupies a significant part of commonsense knowledge. We propose a novel method to mine the deep commonsense knowledge distributed in sentences.
• Score: 25.31716910260664
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Knowledge facts are typically represented by relational triples, while we observe that some commonsense facts are represented by the triples whose forms are inconsistent with the expression of language. This inconsistency puts forward a challenge for pre-trained language models to deal with these commonsense knowledge facts. In this paper, we term such knowledge as deep commonsense knowledge and conduct extensive exploratory experiments on it. We show that deep commonsense knowledge occupies a significant part of commonsense knowledge while conventional methods fail to capture it effectively. We further propose a novel method to mine the deep commonsense knowledge distributed in sentences, alleviating the reliance of conventional methods on the triple representation form of knowledge. Experiments demonstrate that the proposal significantly improves the performance in mining deep commonsense knowledge.

Related papers

• What Really is Commonsense Knowledge? [58.5342212738895]
  We survey existing definitions of commonsense knowledge, ground into the three frameworks for defining concepts, and consolidate them into a unified definition of commonsense knowledge. We then use the consolidated definition for annotations and experiments on the CommonsenseQA and CommonsenseQA 2.0 datasets. Our study shows that there exists a large portion of non-commonsense-knowledge instances in the two datasets, and a large performance gap on these two subsets.
  arXiv  Detail & Related papers  (2024-11-06T14:54:19Z)
• DiffuCOMET: Contextual Commonsense Knowledge Diffusion [29.23102821128395]
  In this work, we develop a series of knowledge models, DiffuCOMET, that leverage diffusion to learn to reconstruct the implicit semantic connections between narrative contexts and relevant commonsense knowledge. To evaluate DiffuCOMET, we introduce new metrics for commonsense inference that more closely measure knowledge diversity and contextual relevance. Our results on two different benchmarks, ComFact and WebNLG+, show that knowledge generated by DiffuCOMET achieves a better trade-off between commonsense diversity, contextual relevance and alignment to known gold references.
  arXiv  Detail & Related papers  (2024-02-26T20:35:34Z)
• Commonsense Knowledge Transfer for Pre-trained Language Models [83.01121484432801]
  We introduce commonsense knowledge transfer, a framework to transfer the commonsense knowledge stored in a neural commonsense knowledge model to a general-purpose pre-trained language model. It first exploits general texts to form queries for extracting commonsense knowledge from the neural commonsense knowledge model. It then refines the language model with two self-supervised objectives: commonsense mask infilling and commonsense relation prediction.
  arXiv  Detail & Related papers  (2023-06-04T15:44:51Z)
• Investigating Forgetting in Pre-Trained Representations Through Continual Learning [51.30807066570425]
  We study the effect of representation forgetting on the generality of pre-trained language models. We find that the generality is destructed in various pre-trained LMs, and syntactic and semantic knowledge is forgotten through continual learning.
  arXiv  Detail & Related papers  (2023-05-10T08:27:59Z)
• Eliciting Knowledge from Large Pre-Trained Models for Unsupervised Knowledge-Grounded Conversation [45.95864432188745]
  Recent advances in large-scale pre-training provide large models with the potential to learn knowledge from the raw text. We propose various methods that best elicit knowledge from large models. Our human study indicates that, though hallucinations exist, large models post the unique advantage of being able to output common sense.
  arXiv  Detail & Related papers  (2022-11-03T04:48:38Z)
• ComFact: A Benchmark for Linking Contextual Commonsense Knowledge [31.19689856957576]
  We propose the new task of commonsense fact linking, where models are given contexts and trained to identify situationally-relevant commonsense knowledge from KGs. Our novel benchmark, ComFact, contains 293k in-context relevance annotations for commonsense across four stylistically diverse datasets.
  arXiv  Detail & Related papers  (2022-10-23T09:30:39Z)
• Generated Knowledge Prompting for Commonsense Reasoning [53.88983683513114]
  We propose generating knowledge statements directly from a language model with a generic prompt format. This approach improves performance of both off-the-shelf and finetuned language models on four commonsense reasoning tasks. Notably, we find that a model's predictions can improve when using its own generated knowledge.
  arXiv  Detail & Related papers  (2021-10-15T21:58:03Z)
• Dimensions of Commonsense Knowledge [60.49243784752026]
  We survey a wide range of popular commonsense sources with a special focus on their relations. We consolidate these relations into 13 knowledge dimensions, each abstracting over more specific relations found in sources.
  arXiv  Detail & Related papers  (2021-01-12T17:52:39Z)
• Improving Machine Reading Comprehension with Contextualized Commonsense Knowledge [62.46091695615262]
  We aim to extract commonsense knowledge to improve machine reading comprehension. We propose to represent relations implicitly by situating structured knowledge in a context. We employ a teacher-student paradigm to inject multiple types of contextualized knowledge into a student machine reader.
  arXiv  Detail & Related papers  (2020-09-12T17:20:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.