Multi-Step Inference for Reasoning Over Paragraphs

• URL: http://arxiv.org/abs/2004.02995v2
• Date: Mon, 7 Jun 2021 04:09:02 GMT
• Title: Multi-Step Inference for Reasoning Over Paragraphs
• Authors: Jiangming Liu, Matt Gardner, Shay B. Cohen, Mirella Lapata
• Abstract summary: Complex reasoning over text requires understanding and chaining together free-form predicates and logical connectives. We present a compositional model reminiscent of neural module networks that can perform chained logical reasoning.
• Score: 95.91527524872832
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Complex reasoning over text requires understanding and chaining together free-form predicates and logical connectives. Prior work has largely tried to do this either symbolically or with black-box transformers. We present a middle ground between these two extremes: a compositional model reminiscent of neural module networks that can perform chained logical reasoning. This model first finds relevant sentences in the context and then chains them together using neural modules. Our model gives significant performance improvements (up to 29\% relative error reduction when comfibined with a reranker) on ROPES, a recently introduced complex reasoning dataset.

Related papers

• Probabilistic Transformer: A Probabilistic Dependency Model for Contextual Word Representation [52.270712965271656]
  We propose a new model of contextual word representation, not from a neural perspective, but from a purely syntactic and probabilistic perspective. We find that the graph of our model resembles transformers, with correspondences between dependencies and self-attention. Experiments show that our model performs competitively to transformers on small to medium sized datasets.
  arXiv  Detail & Related papers  (2023-11-26T06:56:02Z)
• Emulating the Human Mind: A Neural-symbolic Link Prediction Model with Fast and Slow Reasoning and Filtered Rules [4.979279893937017]
  We introduce a novel Neural-Symbolic model named FaSt-FLiP. Our objective is to combine a logical and neural model for enhanced link prediction.
  arXiv  Detail & Related papers  (2023-10-21T12:45:11Z)
• Exposing Attention Glitches with Flip-Flop Language Modeling [55.0688535574859]
  This work identifies and analyzes the phenomenon of attention glitches in large language models. We introduce flip-flop language modeling (FFLM), a family of synthetic benchmarks designed to probe the extrapolative behavior of neural language models. We find that Transformer FFLMs suffer from a long tail of sporadic reasoning errors, some of which we can eliminate using various regularization techniques.
  arXiv  Detail & Related papers  (2023-06-01T17:44:35Z)
• APOLLO: A Simple Approach for Adaptive Pretraining of Language Models for Logical Reasoning [73.3035118224719]
  We propose APOLLO, an adaptively pretrained language model that has improved logical reasoning abilities. APOLLO performs comparably on ReClor and outperforms baselines on LogiQA.
  arXiv  Detail & Related papers  (2022-12-19T07:40:02Z)
• AdaLoGN: Adaptive Logic Graph Network for Reasoning-Based Machine Reading Comprehension [21.741085513119785]
  Recent machine reading comprehension datasets such as ReClor and LogiQA require performing logical reasoning over text. We present a neural-symbolic approach which, to predict an answer, passes messages over a graph representing logical relations between text units.
  arXiv  Detail & Related papers  (2022-03-16T23:51:01Z)
• Neural Unsupervised Semantic Role Labeling [48.69930912510414]
  We present the first neural unsupervised model for semantic role labeling. We decompose the task as two argument related subtasks, identification and clustering. Experiments on CoNLL-2009 English dataset demonstrate that our model outperforms previous state-of-the-art baseline.
  arXiv  Detail & Related papers  (2021-04-19T04:50:16Z)
• Understanding Neural Abstractive Summarization Models via Uncertainty [54.37665950633147]
  seq2seq abstractive summarization models generate text in a free-form manner. We study the entropy, or uncertainty, of the model's token-level predictions. We show that uncertainty is a useful perspective for analyzing summarization and text generation models more broadly.
  arXiv  Detail & Related papers  (2020-10-15T16:57:27Z)
• Logic Constrained Pointer Networks for Interpretable Textual Similarity [11.142649867439406]
  We introduce a novel pointer network based model with a sentinel gating function to align constituent chunks. We improve this base model with a loss function to equally penalize misalignments in both sentences, ensuring the alignments are bidirectional. The model achieves an F1 score of 97.73 and 96.32 on the benchmark SemEval datasets for the chunk alignment task.
  arXiv  Detail & Related papers  (2020-07-15T13:01:44Z)
• An Analysis of the Utility of Explicit Negative Examples to Improve the Syntactic Abilities of Neural Language Models [32.183409062294466]
  We explore the utilities of explicit negative examples in training neural language models. We find that even with our direct learning signals the models still suffer from resolving agreement across an object-relative clause.
  arXiv  Detail & Related papers  (2020-04-06T07:47:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.