Exploring End-to-End Differentiable Natural Logic Modeling

• URL: http://arxiv.org/abs/2011.04044v1
• Date: Sun, 8 Nov 2020 18:18:15 GMT
• Title: Exploring End-to-End Differentiable Natural Logic Modeling
• Authors: Yufei Feng, Zi'ou Zheng, Quan Liu, Michael Greenspan, Xiaodan Zhu
• Abstract summary: We explore end-to-end trained differentiable models that integrate natural logic with neural networks. The proposed model adapts module networks to model natural logic operations, which is enhanced with a memory component to model contextual information.
• Score: 21.994060519995855
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We explore end-to-end trained differentiable models that integrate natural logic with neural networks, aiming to keep the backbone of natural language reasoning based on the natural logic formalism while introducing subsymbolic vector representations and neural components. The proposed model adapts module networks to model natural logic operations, which is enhanced with a memory component to model contextual information. Experiments show that the proposed framework can effectively model monotonicity-based reasoning, compared to the baseline neural network models without built-in inductive bias for monotonicity-based reasoning. Our proposed model shows to be robust when transferred from upward to downward inference. We perform further analyses on the performance of the proposed model on aggregation, showing the effectiveness of the proposed subcomponents on helping achieve better intermediate aggregation performance.

Related papers

• A Transition System Abstraction Framework for Neural Network Dynamical System Models [2.414910571475855]
  This paper proposes a transition system abstraction framework for neural network dynamical system models. The framework is able to abstract a data-driven neural network model into a transition system, making the neural network model interpretable.
  arXiv  Detail & Related papers  (2024-02-18T23:49:18Z)
• On the Trade-off Between Efficiency and Precision of Neural Abstraction [62.046646433536104]
  Neural abstractions have been recently introduced as formal approximations of complex, nonlinear dynamical models. We employ formal inductive synthesis procedures to generate neural abstractions that result in dynamical models with these semantics.
  arXiv  Detail & Related papers  (2023-07-28T13:22:32Z)
• A Statistical-Modelling Approach to Feedforward Neural Network Model Selection [0.8287206589886881]
  Feedforward neural networks (FNNs) can be viewed as non-linear regression models. A novel model selection method is proposed using the Bayesian information criterion (BIC) for FNNs. The choice of BIC over out-of-sample performance leads to an increased probability of recovering the true model.
  arXiv  Detail & Related papers  (2022-07-09T11:07:04Z)
• On the Generalization and Adaption Performance of Causal Models [99.64022680811281]
  Differentiable causal discovery has proposed to factorize the data generating process into a set of modules. We study the generalization and adaption performance of such modular neural causal models. Our analysis shows that the modular neural causal models outperform other models on both zero and few-shot adaptation in low data regimes.
  arXiv  Detail & Related papers  (2022-06-09T17:12:32Z)
• Neuro-symbolic Natural Logic with Introspective Revision for Natural Language Inference [17.636872632724582]
  We introduce a neuro-symbolic natural logic framework based on reinforcement learning with introspective revision. The proposed model has built-in interpretability and shows superior capability in monotonicity inference, systematic generalization, and interpretability.
  arXiv  Detail & Related papers  (2022-03-09T16:31:58Z)
• Decomposing Natural Logic Inferences in Neural NLI [9.606462437067984]
  We investigate whether neural NLI models capture the crucial semantic features central to natural logic: monotonicity and concept inclusion. We find that monotonicity information is notably weak in the representations of popular NLI models which achieve high scores on benchmarks.
  arXiv  Detail & Related papers  (2021-12-15T17:35:30Z)
• Instance-Based Neural Dependency Parsing [56.63500180843504]
  We develop neural models that possess an interpretable inference process for dependency parsing. Our models adopt instance-based inference, where dependency edges are extracted and labeled by comparing them to edges in a training set.
  arXiv  Detail & Related papers  (2021-09-28T05:30:52Z)
• Causal Abstractions of Neural Networks [9.291492712301569]
  We propose a new structural analysis method grounded in a formal theory of textitcausal abstraction. We apply this method to analyze neural models trained on Multiply Quantified Natural Language Inference (MQNLI) corpus.
  arXiv  Detail & Related papers  (2021-06-06T01:07:43Z)
• Firearm Detection via Convolutional Neural Networks: Comparing a Semantic Segmentation Model Against End-to-End Solutions [68.8204255655161]
  Threat detection of weapons and aggressive behavior from live video can be used for rapid detection and prevention of potentially deadly incidents. One way for achieving this is through the use of artificial intelligence and, in particular, machine learning for image analysis. We compare a traditional monolithic end-to-end deep learning model and a previously proposed model based on an ensemble of simpler neural networks detecting fire-weapons via semantic segmentation.
  arXiv  Detail & Related papers  (2020-12-17T15:19:29Z)
• Explaining and Improving Model Behavior with k Nearest Neighbor Representations [107.24850861390196]
  We propose using k nearest neighbor representations to identify training examples responsible for a model's predictions. We show that kNN representations are effective at uncovering learned spurious associations. Our results indicate that the kNN approach makes the finetuned model more robust to adversarial inputs.
  arXiv  Detail & Related papers  (2020-10-18T16:55:25Z)
• Control as Hybrid Inference [62.997667081978825]
  We present an implementation of CHI which naturally mediates the balance between iterative and amortised inference. We verify the scalability of our algorithm on a continuous control benchmark, demonstrating that it outperforms strong model-free and model-based baselines.
  arXiv  Detail & Related papers  (2020-07-11T19:44:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.