Rule Learning as Machine Translation using the Atomic Knowledge Bank

• URL: http://arxiv.org/abs/2311.02765v1
• Date: Sun, 5 Nov 2023 20:48:54 GMT
• Title: Rule Learning as Machine Translation using the Atomic Knowledge Bank
• Authors: Kristoffer {\AE}s{\o}y and Ana Ozaki
• Abstract summary: We explore the capability of transformers to translate sentences expressing rules in natural language into logical rules. We perform experiments using the DKET dataset from the literature and create a dataset for language to logic translation based on the Atomic knowledge bank.
• Score: 8.9969167872226
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Machine learning models, and in particular language models, are being applied to various tasks that require reasoning. While such models are good at capturing patterns their ability to reason in a trustable and controlled manner is frequently questioned. On the other hand, logic-based rule systems allow for controlled inspection and already established verification methods. However it is well-known that creating such systems manually is time-consuming and prone to errors. We explore the capability of transformers to translate sentences expressing rules in natural language into logical rules. We see reasoners as the most reliable tools for performing logical reasoning and focus on translating language into the format expected by such tools. We perform experiments using the DKET dataset from the literature and create a dataset for language to logic translation based on the Atomic knowledge bank.

Related papers

• Empower Nested Boolean Logic via Self-Supervised Curriculum Learning [67.46052028752327]
  We find that any pre-trained language models even including large language models only behave like a random selector in the face of multi-nested logic. To empower language models with this fundamental capability, this paper proposes a new self-supervised learning method textitCurriculum Logical Reasoning (textscClr)
  arXiv  Detail & Related papers  (2023-10-09T06:54:02Z)
• Planning with Logical Graph-based Language Model for Instruction Generation [9.70880913062245]
  We propose a graph-based language model, Logical-GLM, to infuse logic into language models. We generate logical skeletons to guide language model training, infusing domain knowledge into language models. Our approach can generate instructional texts with more correct logic owing to the internalized domain knowledge.
  arXiv  Detail & Related papers  (2023-08-26T06:28:14Z)
• Language Models as Inductive Reasoners [125.99461874008703]
  We propose a new paradigm (task) for inductive reasoning, which is to induce natural language rules from natural language facts. We create a dataset termed DEER containing 1.2k rule-fact pairs for the task, where rules and facts are written in natural language. We provide the first and comprehensive analysis of how well pretrained language models can induce natural language rules from natural language facts.
  arXiv  Detail & Related papers  (2022-12-21T11:12:14Z)
• APOLLO: A Simple Approach for Adaptive Pretraining of Language Models for Logical Reasoning [73.3035118224719]
  We propose APOLLO, an adaptively pretrained language model that has improved logical reasoning abilities. APOLLO performs comparably on ReClor and outperforms baselines on LogiQA.
  arXiv  Detail & Related papers  (2022-12-19T07:40:02Z)
• Learning Symbolic Rules for Reasoning in Quasi-Natural Language [74.96601852906328]
  We build a rule-based system that can reason with natural language input but without the manual construction of rules. We propose MetaQNL, a "Quasi-Natural" language that can express both formal logic and natural language sentences. Our approach achieves state-of-the-art accuracy on multiple reasoning benchmarks.
  arXiv  Detail & Related papers  (2021-11-23T17:49:00Z)
• BERT got a Date: Introducing Transformers to Temporal Tagging [4.651578365545765]
  We present a transformer encoder-decoder model using the RoBERTa language model as our best performing system. Our model surpasses previous works in temporal tagging and type classification, especially on rare classes.
  arXiv  Detail & Related papers  (2021-09-30T08:54:21Z)
• Leap-Of-Thought: Teaching Pre-Trained Models to Systematically Reason Over Implicit Knowledge [96.92252296244233]
  Large pre-trained language models (LMs) acquire some reasoning capacity, but this ability is difficult to control. We show that LMs can be trained to reliably perform systematic reasoning combining both implicit, pre-trained knowledge and explicit natural language statements. Our work paves a path towards open-domain systems that constantly improve by interacting with users who can instantly correct a model by adding simple natural language statements.
  arXiv  Detail & Related papers  (2020-06-11T17:02:20Z)
• How Context Affects Language Models' Factual Predictions [134.29166998377187]
  We integrate information from a retrieval system with a pre-trained language model in a purely unsupervised way. We report that augmenting pre-trained language models in this way dramatically improves performance and that the resulting system, despite being unsupervised, is competitive with a supervised machine reading baseline.
  arXiv  Detail & Related papers  (2020-05-10T09:28:12Z)
• Transformers as Soft Reasoners over Language [33.291806251021185]
  This paper investigates a problem where the facts and rules are provided as natural language sentences, thus bypassing a formal representation. We train transformers to emulate reason (or reasoning) over these sentences using synthetically generated data. Our models, that we call RuleTakers, provide the first empirical demonstration that this kind of soft reasoning over language is learnable.
  arXiv  Detail & Related papers  (2020-02-14T04:23:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.