Scalable Coupling of Deep Learning with Logical Reasoning

• URL: http://arxiv.org/abs/2305.07617v2
• Date: Tue, 18 Jul 2023 09:40:34 GMT
• Title: Scalable Coupling of Deep Learning with Logical Reasoning
• Authors: Marianne Defresne, Sophie Barbe, Thomas Schiex
• Abstract summary: We introduce a scalable neural architecture and loss function dedicated to learning the constraints and criteria of NP-hard reasoning problems. Our loss function solves one of the main limitations of Besag's pseudo-loglikelihood, enabling learning of high energies.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: In the ongoing quest for hybridizing discrete reasoning with neural nets, there is an increasing interest in neural architectures that can learn how to solve discrete reasoning or optimization problems from natural inputs. In this paper, we introduce a scalable neural architecture and loss function dedicated to learning the constraints and criteria of NP-hard reasoning problems expressed as discrete Graphical Models. Our loss function solves one of the main limitations of Besag's pseudo-loglikelihood, enabling learning of high energies. We empirically show it is able to efficiently learn how to solve NP-hard reasoning problems from natural inputs as the symbolic, visual or many-solutions Sudoku problems as well as the energy optimization formulation of the protein design problem, providing data efficiency, interpretability, and \textit{a posteriori} control over predictions.

Related papers

• Neural Algorithmic Reasoning for Combinatorial Optimisation [20.36694807847833]
  We propose leveraging recent advancements in neural reasoning to improve the learning of CO problems. We suggest pre-training our neural model on relevant algorithms before training it on CO instances. Our results demonstrate that by using this learning setup, we achieve superior performance compared to non-algorithmically informed deep learning models.
  arXiv  Detail & Related papers  (2023-05-18T13:59:02Z)
• Knowledge-based Analogical Reasoning in Neuro-symbolic Latent Spaces [20.260546238369205]
  We propose a framework that combines the pattern recognition abilities of neural networks with symbolic reasoning and background knowledge. We take inspiration from the 'neural algorithmic reasoning' approach [DeepMind 2020] and use problem-specific background knowledge. We test this on visual analogy problems in RAVENs Progressive Matrices, and achieve accuracy competitive with human performance.
  arXiv  Detail & Related papers  (2022-09-19T04:03:20Z)
• Learning Iterative Reasoning through Energy Minimization [77.33859525900334]
  We present a new framework for iterative reasoning with neural networks. We train a neural network to parameterize an energy landscape over all outputs. We implement each step of the iterative reasoning as an energy minimization step to find a minimal energy solution.
  arXiv  Detail & Related papers  (2022-06-30T17:44:20Z)
• Generalization of Neural Combinatorial Solvers Through the Lens of Adversarial Robustness [68.97830259849086]
  Most datasets only capture a simpler subproblem and likely suffer from spurious features. We study adversarial robustness - a local generalization property - to reveal hard, model-specific instances and spurious features. Unlike in other applications, where perturbation models are designed around subjective notions of imperceptibility, our perturbation models are efficient and sound. Surprisingly, with such perturbations, a sufficiently expressive neural solver does not suffer from the limitations of the accuracy-robustness trade-off common in supervised learning.
  arXiv  Detail & Related papers  (2021-10-21T07:28:11Z)
• CombOptNet: Fit the Right NP-Hard Problem by Learning Integer Programming Constraints [20.659237363210774]
  We aim to integrate integer programming solvers into neural network architectures as layers capable of learning both the cost terms and the constraints. The resulting end-to-end trainable architectures jointly extract features from raw data and solve a suitable (learned) problem with state-of-the-art integer programming solvers.
  arXiv  Detail & Related papers  (2021-05-05T21:52:53Z)
• A neural anisotropic view of underspecification in deep learning [60.119023683371736]
  We show that the way neural networks handle the underspecification of problems is highly dependent on the data representation. Our results highlight that understanding the architectural inductive bias in deep learning is fundamental to address the fairness, robustness, and generalization of these systems.
  arXiv  Detail & Related papers  (2021-04-29T14:31:09Z)
• Conditional physics informed neural networks [85.48030573849712]
  We introduce conditional PINNs (physics informed neural networks) for estimating the solution of classes of eigenvalue problems. We show that a single deep neural network can learn the solution of partial differential equations for an entire class of problems.
  arXiv  Detail & Related papers  (2021-04-06T18:29:14Z)
• SMART: A Situation Model for Algebra Story Problems via Attributed Grammar [74.1315776256292]
  We introduce the concept of a emphsituation model, which originates from psychology studies to represent the mental states of humans in problem-solving. We show that the proposed model outperforms all previous neural solvers by a large margin while preserving much better interpretability.
  arXiv  Detail & Related papers  (2020-12-27T21:03:40Z)
• Neural Network Approximations of Compositional Functions With Applications to Dynamical Systems [3.660098145214465]
  We develop an approximation theory for compositional functions and their neural network approximations. We identify a set of key features of compositional functions and the relationship between the features and the complexity of neural networks. In addition to function approximations, we prove several formulae of error upper bounds for neural networks.
  arXiv  Detail & Related papers  (2020-12-03T04:40:25Z)
• Neural Logic Reasoning [47.622957656745356]
  We propose Logic-Integrated Neural Network (LINN) to integrate the power of deep learning and logic reasoning. LINN learns basic logical operations such as AND, OR, NOT as neural modules, and conducts propositional logical reasoning through the network for inference. Experiments show that LINN significantly outperforms state-of-the-art recommendation models in Top-K recommendation.
  arXiv  Detail & Related papers  (2020-08-20T14:53:23Z)
• Relational Neural Machines [19.569025323453257]
  This paper presents a novel framework allowing jointly train the parameters of the learners and of a First-Order Logic based reasoner. A Neural Machine is able recover both classical learning results in case of pure sub-symbolic learning, and Markov Logic Networks. Proper algorithmic solutions are devised to make learning and inference tractable in large-scale problems.
  arXiv  Detail & Related papers  (2020-02-06T10:53:57Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.