Language Models as Inductive Reasoners

• URL: http://arxiv.org/abs/2212.10923v3
• Date: Mon, 5 Feb 2024 11:19:24 GMT
• Title: Language Models as Inductive Reasoners
• Authors: Zonglin Yang, Li Dong, Xinya Du, Hao Cheng, Erik Cambria, Xiaodong Liu, Jianfeng Gao, Furu Wei
• Abstract summary: We propose a new paradigm (task) for inductive reasoning, which is to induce natural language rules from natural language facts. We create a dataset termed DEER containing 1.2k rule-fact pairs for the task, where rules and facts are written in natural language. We provide the first and comprehensive analysis of how well pretrained language models can induce natural language rules from natural language facts.
• Score: 125.99461874008703
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Inductive reasoning is a core component of human intelligence. In the past research of inductive reasoning within computer science, formal language is used as representations of knowledge (facts and rules, more specifically). However, formal language can cause systematic problems for inductive reasoning such as disability of handling raw input such as natural language, sensitiveness to mislabeled data, and incapacity to handle ambiguous input. To this end, we propose a new paradigm (task) for inductive reasoning, which is to induce natural language rules from natural language facts, and create a dataset termed DEER containing 1.2k rule-fact pairs for the task, where rules and facts are written in natural language. New automatic metrics are also proposed and analysed for the evaluation of this task. With DEER, we investigate a modern approach for inductive reasoning where we use natural language as representation for knowledge instead of formal language and use pretrained language models as ''reasoners''. Moreover, we provide the first and comprehensive analysis of how well pretrained language models can induce natural language rules from natural language facts. We also propose a new framework drawing insights from philosophy literature for this task, which we show in the experiment section that surpasses baselines in both automatic and human evaluations. We discuss about our future perspectives for inductive reasoning in Section 7. Dataset and code are available at https://github.com/ZonglinY/Inductive_Reasoning.

Related papers

• Language Models can be Logical Solvers [99.40649402395725]
  We introduce LoGiPT, a novel language model that directly emulates the reasoning processes of logical solvers. LoGiPT is fine-tuned on a newly constructed instruction-tuning dataset derived from revealing and refining the invisible reasoning process of deductive solvers.
  arXiv  Detail & Related papers  (2023-11-10T16:23:50Z)
• Natural Language Decompositions of Implicit Content Enable Better Text Representations [56.85319224208865]
  We introduce a method for the analysis of text that takes implicitly communicated content explicitly into account. We use a large language model to produce sets of propositions that are inferentially related to the text that has been observed. Our results suggest that modeling the meanings behind observed language, rather than the literal text alone, is a valuable direction for NLP.
  arXiv  Detail & Related papers  (2023-05-23T23:45:20Z)
• Injecting structural hints: Using language models to study inductive biases in language learning [40.8902073270634]
  We inject inductive bias into language models by pretraining on formally-structured data. We then evaluate the biased learners' ability to learn typologically-diverse natural languages. We show that non-context-free relationships form the best inductive biases.
  arXiv  Detail & Related papers  (2023-04-25T18:00:08Z)
• ChatABL: Abductive Learning via Natural Language Interaction with ChatGPT [72.83383437501577]
  Large language models (LLMs) have recently demonstrated significant potential in mathematical abilities. LLMs currently have difficulty in bridging perception, language understanding and reasoning capabilities. This paper presents a novel method for integrating LLMs into the abductive learning framework.
  arXiv  Detail & Related papers  (2023-04-21T16:23:47Z)
• Logical Reasoning over Natural Language as Knowledge Representation: A Survey [43.29703101875716]
  This paper provides an overview on a new paradigm of logical reasoning, which uses natural language as knowledge representation and pretrained language models as reasoners. This new paradigm is promising since it not only alleviates many challenges of formal representation but also has advantages over end-to-end neural methods.
  arXiv  Detail & Related papers  (2023-03-21T16:56:05Z)
• Transparency Helps Reveal When Language Models Learn Meaning [71.96920839263457]
  Our systematic experiments with synthetic data reveal that, with languages where all expressions have context-independent denotations, both autoregressive and masked language models learn to emulate semantic relations between expressions. Turning to natural language, our experiments with a specific phenomenon -- referential opacity -- add to the growing body of evidence that current language models do not well-represent natural language semantics.
  arXiv  Detail & Related papers  (2022-10-14T02:35:19Z)
• Learning Symbolic Rules for Reasoning in Quasi-Natural Language [74.96601852906328]
  We build a rule-based system that can reason with natural language input but without the manual construction of rules. We propose MetaQNL, a "Quasi-Natural" language that can express both formal logic and natural language sentences. Our approach achieves state-of-the-art accuracy on multiple reasoning benchmarks.
  arXiv  Detail & Related papers  (2021-11-23T17:49:00Z)
• Towards Coinductive Models for Natural Language Understanding. Bringing together Deep Learning and Deep Semantics [0.0]
  Coinduction has been successfully used in the design of operating systems and programming languages. It has been present in text mining, machine translation, and in some attempts to model intensionality and modalities. This article shows several examples of the joint appearance of induction and coinduction in natural language processing.
  arXiv  Detail & Related papers  (2020-12-09T03:10:36Z)
• Universal linguistic inductive biases via meta-learning [36.43388942327124]
  It is unclear which inductive biases can explain observed patterns in language acquisition. We introduce a framework for giving linguistic inductive biases to a neural network model. We demonstrate this framework with a case study based on syllable structure.
  arXiv  Detail & Related papers  (2020-06-29T19:15:10Z)
• A Dataset for Statutory Reasoning in Tax Law Entailment and Question Answering [37.66486350122862]
  This paper investigates the performance of natural language understanding approaches on statutory reasoning. We introduce a dataset, together with a legal-domain text corpus. We contrast this with a hand-constructed Prolog-based system, designed to fully solve the task.
  arXiv  Detail & Related papers  (2020-05-11T16:54:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.