Mind Reasoning Manners: Enhancing Type Perception for Generalized Zero-shot Logical Reasoning over Text

• URL: http://arxiv.org/abs/2301.02983v1
• Date: Sun, 8 Jan 2023 05:24:34 GMT
• Title: Mind Reasoning Manners: Enhancing Type Perception for Generalized Zero-shot Logical Reasoning over Text
• Authors: Fangzhi Xu, Jun Liu, Qika Lin, Tianzhe Zhao, Jian Zhang, Lingling Zhang
• Abstract summary: We propose a new benchmark for generalized zero-shot logical reasoning, named ZsLR. For problem 1, we propose a new benchmark for generalized zero-shot logical reasoning, named ZsLR. For problem 2, a type-aware model TaCo is proposed to improve the type perception in the global representation.
• Score: 12.988062333041398
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Logical reasoning task involves diverse types of complex reasoning over text, based on the form of multiple-choice question answering. Given the context, question and a set of options as the input, previous methods achieve superior performances on the full-data setting. However, the current benchmark dataset has the ideal assumption that the reasoning type distribution on the train split is close to the test split, which is inconsistent with many real application scenarios. To address it, there remain two problems to be studied: (1) How is the zero-shot capability of the models (train on seen types and test on unseen types)? (2) How to enhance the perception of reasoning types for the models? For problem 1, we propose a new benchmark for generalized zero-shot logical reasoning, named ZsLR. It includes six splits based on the three type sampling strategies. For problem 2, a type-aware model TaCo is proposed. It utilizes both the heuristic input reconstruction and the contrastive learning to improve the type perception in the global representation. Extensive experiments on both the zero-shot and full-data settings prove the superiority of TaCo over the state-of-the-art methods. Also, we experiment and verify the generalization capability of TaCo on other logical reasoning dataset.

Related papers

• Uncovering Autoregressive LLM Knowledge of Thematic Fit in Event Representation [0.09558392439655014]
  We assess whether pre-trained autoregressive LLMs possess consistent, expressible knowledge about thematic fit. We evaluate both closed and open state-of-the-art LLMs on several psycholinguistic datasets. Our results show that chain-of-thought reasoning is more effective on datasets with self-explanatory semantic role labels.
  arXiv  Detail & Related papers  (2024-10-19T18:25:30Z)
• Instance-adaptive Zero-shot Chain-of-Thought Prompting [32.700073951068575]
  Zero-shot Chain-of-Thought (CoT) prompting emerges as a simple and effective strategy for enhancing the performance of large language models (LLMs) in real-world reasoning tasks. This work introduces an instance-adaptive prompting algorithm as an alternative zero-shot CoT reasoning scheme by adaptively differentiating good and bad prompts.
  arXiv  Detail & Related papers  (2024-09-30T16:00:34Z)
• Thought-Path Contrastive Learning via Premise-Oriented Data Augmentation for Logical Reading Comprehension [9.67774998354062]
  Previous research has primarily focused on enhancing logical reasoning capabilities through Chain-of-Thought (CoT) or data augmentation. We propose a Premise-Oriented Data Augmentation (PODA) framework to generate CoT rationales including analyses for both correct and incorrect options. We also introduce a novel thought-path contrastive learning method that compares reasoning paths between the original and counterfactual samples.
  arXiv  Detail & Related papers  (2024-09-22T15:44:43Z)
• Adapting Vision-Language Models to Open Classes via Test-Time Prompt Tuning [50.26965628047682]
  Adapting pre-trained models to open classes is a challenging problem in machine learning. In this paper, we consider combining the advantages of both and come up with a test-time prompt tuning approach. Our proposed method outperforms all comparison methods on average considering both base and new classes.
  arXiv  Detail & Related papers  (2024-08-29T12:34:01Z)
• General Purpose Verification for Chain of Thought Prompting [16.381123651223763]
  We explore ways to improve reasoning capabilities of Large Language Models (LLMs) We propose three general principles that a model should adhere to while reasoning. We apply these constraints to the reasoning steps generated by the LLM to improve the accuracy of the final generation.
  arXiv  Detail & Related papers  (2024-04-30T21:15:17Z)
• Explanation Selection Using Unlabeled Data for Chain-of-Thought Prompting [80.9896041501715]
  Explanations that have not been "tuned" for a task, such as off-the-shelf explanations written by nonexperts, may lead to mediocre performance. This paper tackles the problem of how to optimize explanation-infused prompts in a blackbox fashion.
  arXiv  Detail & Related papers  (2023-02-09T18:02:34Z)
• Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering [124.16250115608604]
  We present Science Question Answering (SQA), a new benchmark that consists of 21k multimodal multiple choice questions with a diverse set of science topics and annotations of their answers with corresponding lectures and explanations. We show that SQA improves the question answering performance by 1.20% in few-shot GPT-3 and 3.99% in fine-tuned UnifiedQA. Our analysis further shows that language models, similar to humans, benefit from explanations to learn from fewer data and achieve the same performance with just 40% of the data.
  arXiv  Detail & Related papers  (2022-09-20T07:04:24Z)
• Logical Satisfiability of Counterfactuals for Faithful Explanations in NLI [60.142926537264714]
  We introduce the methodology of Faithfulness-through-Counterfactuals. It generates a counterfactual hypothesis based on the logical predicates expressed in the explanation. It then evaluates if the model's prediction on the counterfactual is consistent with that expressed logic.
  arXiv  Detail & Related papers  (2022-05-25T03:40:59Z)
• Does entity abstraction help generative Transformers reason? [8.159805544989359]
  We study the utility of incorporating entity type abstractions into pre-trained Transformers. We test these methods on four NLP tasks requiring different forms of logical reasoning.
  arXiv  Detail & Related papers  (2022-01-05T19:00:53Z)
• Contrastive Explanations for Model Interpretability [77.92370750072831]
  We propose a methodology to produce contrastive explanations for classification models. Our method is based on projecting model representation to a latent space. Our findings shed light on the ability of label-contrastive explanations to provide a more accurate and finer-grained interpretability of a model's decision.
  arXiv  Detail & Related papers  (2021-03-02T00:36:45Z)
• L2R2: Leveraging Ranking for Abductive Reasoning [65.40375542988416]
  The abductive natural language inference task ($alpha$NLI) is proposed to evaluate the abductive reasoning ability of a learning system. A novel $L2R2$ approach is proposed under the learning-to-rank framework. Experiments on the ART dataset reach the state-of-the-art in the public leaderboard.
  arXiv  Detail & Related papers  (2020-05-22T15:01:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.