OPERA:Operation-Pivoted Discrete Reasoning over Text
- URL: http://arxiv.org/abs/2204.14166v1
- Date: Fri, 29 Apr 2022 15:41:47 GMT
- Title: OPERA:Operation-Pivoted Discrete Reasoning over Text
- Authors: Yongwei Zhou, Junwei Bao, Chaoqun Duan, Haipeng Sun, Jiahui Liang,
Yifan Wang, Jing Zhao, Youzheng Wu, Xiaodong He, Tiejun Zhao
- Abstract summary: OPERA is an operation-pivoted discrete reasoning framework for machine reading comprehension.
It uses lightweight symbolic operations as neural modules to facilitate the reasoning ability and interpretability.
Experiments on both DROP and RACENum datasets show the reasoning ability of OPERA.
- Score: 33.36388276371693
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Machine reading comprehension (MRC) that requires discrete reasoning
involving symbolic operations, e.g., addition, sorting, and counting, is a
challenging task. According to this nature, semantic parsing-based methods
predict interpretable but complex logical forms. However, logical form
generation is nontrivial and even a little perturbation in a logical form will
lead to wrong answers. To alleviate this issue, multi-predictor -based methods
are proposed to directly predict different types of answers and achieve
improvements. However, they ignore the utilization of symbolic operations and
encounter a lack of reasoning ability and interpretability. To inherit the
advantages of these two types of methods, we propose OPERA, an
operation-pivoted discrete reasoning framework, where lightweight symbolic
operations (compared with logical forms) as neural modules are utilized to
facilitate the reasoning ability and interpretability. Specifically, operations
are first selected and then softly executed to simulate the answer reasoning
procedure. Extensive experiments on both DROP and RACENum datasets show the
reasoning ability of OPERA. Moreover, further analysis verifies its
interpretability.
Related papers
- Distributional reasoning in LLMs: Parallel reasoning processes in multi-hop reasoning [8.609587510471943]
We introduce a novel and interpretable analysis of internal multi-hop reasoning processes in large language models.
We show that during inference, the middle layers of the network generate highly interpretable embeddings.
Our findings can help uncover the strategies that LLMs use to solve reasoning tasks, offering insights into the types of thought processes that can emerge from artificial intelligence.
arXiv Detail & Related papers (2024-06-19T21:36:40Z) - LogicBench: Towards Systematic Evaluation of Logical Reasoning Ability of Large Language Models [52.03659714625452]
Recently developed large language models (LLMs) have been shown to perform remarkably well on a wide range of language understanding tasks.
But, can they really "reason" over the natural language?
This question has been receiving significant research attention and many reasoning skills such as commonsense, numerical, and qualitative have been studied.
arXiv Detail & Related papers (2024-04-23T21:08:49Z) - Language Models as Compilers: Simulating Pseudocode Execution Improves Algorithmic Reasoning in Language Models [17.76252625790628]
This paper presents Think-and-Execute, a framework that decomposes the reasoning process of language models into two steps.
With extensive experiments on seven algorithmic reasoning tasks, we demonstrate the effectiveness of Think-and-Execute.
arXiv Detail & Related papers (2024-04-03T08:49:11Z) - Language Models can be Logical Solvers [99.40649402395725]
We introduce LoGiPT, a novel language model that directly emulates the reasoning processes of logical solvers.
LoGiPT is fine-tuned on a newly constructed instruction-tuning dataset derived from revealing and refining the invisible reasoning process of deductive solvers.
arXiv Detail & Related papers (2023-11-10T16:23:50Z) - Representation Synthesis by Probabilistic Many-Valued Logic Operation in
Self-Supervised Learning [10.49712834719005]
We propose a new self-supervised learning (SSL) method for representations that enable logic operations.
Our method can generate a representation that has the features of both representations or only those features common to both representations.
Experiments on image retrieval using MNIST and PascalVOC showed that the representations of our method can be operated by OR and AND operations.
arXiv Detail & Related papers (2023-09-08T06:24:44Z) - APOLLO: A Simple Approach for Adaptive Pretraining of Language Models
for Logical Reasoning [73.3035118224719]
We propose APOLLO, an adaptively pretrained language model that has improved logical reasoning abilities.
APOLLO performs comparably on ReClor and outperforms baselines on LogiQA.
arXiv Detail & Related papers (2022-12-19T07:40:02Z) - MetaLogic: Logical Reasoning Explanations with Fine-Grained Structure [129.8481568648651]
We propose a benchmark to investigate models' logical reasoning capabilities in complex real-life scenarios.
Based on the multi-hop chain of reasoning, the explanation form includes three main components.
We evaluate the current best models' performance on this new explanation form.
arXiv Detail & Related papers (2022-10-22T16:01:13Z) - Prediction or Comparison: Toward Interpretable Qualitative Reasoning [16.02199526395448]
Current approaches use either semantics to transform natural language inputs into logical expressions or a "black-box" model to solve them in one step.
In this work, we categorize qualitative reasoning tasks into two types: prediction and comparison.
In particular, we adopt neural network modules trained in an end-to-end manner to simulate the two reasoning processes.
arXiv Detail & Related papers (2021-06-04T10:27:55Z) - Logic-Driven Context Extension and Data Augmentation for Logical
Reasoning of Text [65.24325614642223]
We propose to understand logical symbols and expressions in the text to arrive at the answer.
Based on such logical information, we put forward a context extension framework and a data augmentation algorithm.
Our method achieves the state-of-the-art performance, and both logic-driven context extension framework and data augmentation algorithm can help improve the accuracy.
arXiv Detail & Related papers (2021-05-08T10:09:36Z) - AR-LSAT: Investigating Analytical Reasoning of Text [57.1542673852013]
We study the challenge of analytical reasoning of text and introduce a new dataset consisting of questions from the Law School Admission Test from 1991 to 2016.
We analyze what knowledge understanding and reasoning abilities are required to do well on this task.
arXiv Detail & Related papers (2021-04-14T02:53:32Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.