Probing the Natural Language Inference Task with Automated Reasoning Tools

• URL: http://arxiv.org/abs/2005.02573v1
• Date: Wed, 6 May 2020 03:18:11 GMT
• Title: Probing the Natural Language Inference Task with Automated Reasoning Tools
• Authors: Zaid Marji, Animesh Nighojkar, John Licato
• Abstract summary: The Natural Language Inference (NLI) task is an important task in modern NLP, as it asks a broad question to which many other tasks may be reducible. We use other techniques to examine the logical structure of the NLI task. We show how well a machine-oriented controlled natural language can be used to parse NLI sentences, and how well automated theorem provers can reason over the resulting formulae.
• Score: 6.445605125467574
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The Natural Language Inference (NLI) task is an important task in modern NLP, as it asks a broad question to which many other tasks may be reducible: Given a pair of sentences, does the first entail the second? Although the state-of-the-art on current benchmark datasets for NLI are deep learning-based, it is worthwhile to use other techniques to examine the logical structure of the NLI task. We do so by testing how well a machine-oriented controlled natural language (Attempto Controlled English) can be used to parse NLI sentences, and how well automated theorem provers can reason over the resulting formulae. To improve performance, we develop a set of syntactic and semantic transformation rules. We report their performance, and discuss implications for NLI and logic-based NLP.

Related papers

• A Novel Cartography-Based Curriculum Learning Method Applied on RoNLI: The First Romanian Natural Language Inference Corpus [71.77214818319054]
  Natural language inference is a proxy for natural language understanding. There is no publicly available NLI corpus for the Romanian language. We introduce the first Romanian NLI corpus (RoNLI) comprising 58K training sentence pairs.
  arXiv  Detail & Related papers  (2024-05-20T08:41:15Z)
• kNN-ICL: Compositional Task-Oriented Parsing Generalization with Nearest Neighbor In-Context Learning [50.40636157214161]
  Task-Oriented Parsing (TOP) enables conversational assistants to interpret user commands expressed in natural language. LLMs have achieved impressive performance in computer programs based on a natural language prompt. This paper focuses on harnessing the capabilities of LLMs for semantic parsing tasks.
  arXiv  Detail & Related papers  (2023-12-17T17:26:50Z)
• Native Language Identification with Large Language Models [60.80452362519818]
  We show that GPT models are proficient at NLI classification, with GPT-4 setting a new performance record of 91.7% on the benchmark11 test set in a zero-shot setting. We also show that unlike previous fully-supervised settings, LLMs can perform NLI without being limited to a set of known classes.
  arXiv  Detail & Related papers  (2023-12-13T00:52:15Z)
• Deep Natural Language Feature Learning for Interpretable Prediction [1.6114012813668932]
  We propose a method to break down a main complex task into a set of intermediary easier sub-tasks. Our method allows for representing each example by a vector consisting of the answers to these questions. We have successfully applied this method to two completely different tasks: detecting incoherence in students' answers to open-ended mathematics exam questions, and screening abstracts for a systematic literature review of scientific papers on climate change and agroecology.
  arXiv  Detail & Related papers  (2023-11-09T21:43:27Z)
• Unravelling Interlanguage Facts via Explainable Machine Learning [10.71581852108984]
  We focus on the internals of an NLI classifier trained by an emphexplainable machine learning algorithm. We use this perspective in order to tackle both NLI and a companion task, guessing whether a text has been written by a native or a non-native speaker. We investigate which kind of linguistic traits are most effective for solving our two tasks, namely, are most indicative of a speaker's L1.
  arXiv  Detail & Related papers  (2022-08-02T14:05:15Z)
• AdaPrompt: Adaptive Model Training for Prompt-based NLP [77.12071707955889]
  We propose AdaPrompt, adaptively retrieving external data for continual pretraining of PLMs. Experimental results on five NLP benchmarks show that AdaPrompt can improve over standard PLMs in few-shot settings. In zero-shot settings, our method outperforms standard prompt-based methods by up to 26.35% relative error reduction.
  arXiv  Detail & Related papers  (2022-02-10T04:04:57Z)
• Polish Natural Language Inference and Factivity -- an Expert-based Dataset and Benchmarks [0.07734726150561087]
  The dataset contains entirely natural language utterances in Polish. It is a representative sample in regards to frequency of main verbs and other linguistic features. BERT-based models consuming only the input sentences show that they capture most of the complexity of NLI/factivity.
  arXiv  Detail & Related papers  (2022-01-10T18:32:55Z)
• NeuralLog: Natural Language Inference with Joint Neural and Logical Reasoning [6.795509403707242]
  We propose an inference framework called NeuralLog, which utilizes both a monotonicity-based logical inference engine and a neural network language model for phrase alignment. Our framework models the NLI task as a classic search problem and uses the beam search algorithm to search for optimal inference paths. Experiments show that our joint logic and neural inference system improves accuracy on the NLI task and can achieve state-of-art accuracy on the SICK and MED datasets.
  arXiv  Detail & Related papers  (2021-05-29T01:02:40Z)
• Infusing Finetuning with Semantic Dependencies [62.37697048781823]
  We show that, unlike syntax, semantics is not brought to the surface by today's pretrained models. We then use convolutional graph encoders to explicitly incorporate semantic parses into task-specific finetuning.
  arXiv  Detail & Related papers  (2020-12-10T01:27:24Z)
• Looking Beyond Sentence-Level Natural Language Inference for Downstream Tasks [15.624486319943015]
  In recent years, the Natural Language Inference (NLI) task has garnered significant attention. We study this unfulfilled promise from the lens of two downstream tasks: question answering (QA), and text summarization. We conjecture that a key difference between the NLI datasets and these downstream tasks concerns the length of the premise.
  arXiv  Detail & Related papers  (2020-09-18T21:44:35Z)
• FewJoint: A Few-shot Learning Benchmark for Joint Language Understanding [55.38905499274026]
  Few-shot learning is one of the key future steps in machine learning. FewJoint is a novel Few-Shot Learning benchmark for NLP.
  arXiv  Detail & Related papers  (2020-09-17T08:17:12Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.