Relational Neurosymbolic Markov Models

• URL: http://arxiv.org/abs/2412.13023v1
• Date: Tue, 17 Dec 2024 15:41:51 GMT
• Title: Relational Neurosymbolic Markov Models
• Authors: Lennert De Smet, Gabriele Venturato, Luc De Raedt, Giuseppe Marra,
• Abstract summary: Sequential problems are ubiquitous in AI, such as in reinforcement learning or natural language processing.<n>We introduce neurosymbolic AI (NeSy) which provides a sound formalism to enforce constraints in deep probabilistic models but scales exponentially on sequential problems.<n>We propose a strategy for inference and learning that scales on sequential settings, and that combines approximate Bayesian inference, automated reasoning, and gradient estimation.
• Score: 13.22004615196798
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Sequential problems are ubiquitous in AI, such as in reinforcement learning or natural language processing. State-of-the-art deep sequential models, like transformers, excel in these settings but fail to guarantee the satisfaction of constraints necessary for trustworthy deployment. In contrast, neurosymbolic AI (NeSy) provides a sound formalism to enforce constraints in deep probabilistic models but scales exponentially on sequential problems. To overcome these limitations, we introduce relational neurosymbolic Markov models (NeSy-MMs), a new class of end-to-end differentiable sequential models that integrate and provably satisfy relational logical constraints. We propose a strategy for inference and learning that scales on sequential settings, and that combines approximate Bayesian inference, automated reasoning, and gradient estimation. Our experiments show that NeSy-MMs can solve problems beyond the current state-of-the-art in neurosymbolic AI and still provide strong guarantees with respect to desired properties. Moreover, we show that our models are more interpretable and that constraints can be adapted at test time to out-of-distribution scenarios.

Related papers

• Semi-Explicit Neural DAEs: Learning Long-Horizon Dynamical Systems with Algebraic Constraints [2.66269503676104]
  We propose a method that explicitly enforces algebraic constraints by projecting each ODE step onto the constraint manifold.<n>PNODEs consistently outperform baselines across six benchmark problems achieving a mean constraint violation error below $10-10$.<n>These results show that constraint projection offers a simple strategy for learning physically consistent long-horizon dynamics.
  arXiv  Detail & Related papers  (2025-05-26T20:31:15Z)
• Certified Neural Approximations of Nonlinear Dynamics [52.79163248326912]
  In safety-critical contexts, the use of neural approximations requires formal bounds on their closeness to the underlying system.<n>We propose a novel, adaptive, and parallelizable verification method based on certified first-order models.
  arXiv  Detail & Related papers  (2025-05-21T13:22:20Z)
• Neurosymbolic Diffusion Models [14.424743331071241]
  Neurosymbolic (NeSy) predictors combine neural perception with symbolic reasoning to solve tasks like visual reasoning.<n>Standard NeSy predictors assume conditional independence between the symbols they extract, thus limiting their ability to model interactions and uncertainty.<n>We introduce neurosymbolic diffusion models (NeSyDMs), a new class of NeSy predictors that use discrete diffusion to model dependencies between symbols.
  arXiv  Detail & Related papers  (2025-05-19T14:07:47Z)
• A Novel Framework for Learning Stochastic Representations for Sequence Generation and Recognition [0.0]
  The ability to generate and recognize sequential data is fundamental for autonomous systems operating in dynamic environments. We propose a novel Recurrent Network with Parametric Biases (RNNPB) Our approach provides a framework for modeling temporal patterns and advances the development of robust and systems in artificial intelligence and robotics.
  arXiv  Detail & Related papers  (2024-12-30T07:27:50Z)
• Projected Neural Differential Equations for Learning Constrained Dynamics [3.570367665112327]
  We introduce a new method for constraining neural differential equations based on projection of the learned vector field to the tangent space of the constraint manifold. PNDEs outperform existing methods while requiring fewer hyper parameters. The proposed approach demonstrates significant potential for enhancing the modeling of constrained dynamical systems.
  arXiv  Detail & Related papers  (2024-10-31T06:32:43Z)
• SynthTree: Co-supervised Local Model Synthesis for Explainable Prediction [15.832975722301011]
  We propose a novel method to enhance explainability with minimal accuracy loss. We have developed novel methods for estimating nodes by leveraging AI techniques. Our findings highlight the critical role that statistical methodologies can play in advancing explainable AI.
  arXiv  Detail & Related papers  (2024-06-16T14:43:01Z)
• Understanding the differences in Foundation Models: Attention, State Space Models, and Recurrent Neural Networks [50.29356570858905]
  We introduce the Dynamical Systems Framework (DSF), which allows a principled investigation of all these architectures in a common representation.<n>We provide principled comparisons between softmax attention and other model classes, discussing the theoretical conditions under which softmax attention can be approximated.<n>This shows the DSF's potential to guide the systematic development of future more efficient and scalable foundation models.
  arXiv  Detail & Related papers  (2024-05-24T17:19:57Z)
• The Role of Foundation Models in Neuro-Symbolic Learning and Reasoning [54.56905063752427]
  Neuro-Symbolic AI (NeSy) holds promise to ensure the safe deployment of AI systems. Existing pipelines that train the neural and symbolic components sequentially require extensive labelling. New architecture, NeSyGPT, fine-tunes a vision-language foundation model to extract symbolic features from raw data.
  arXiv  Detail & Related papers  (2024-02-02T20:33:14Z)
• On the Trade-off Between Efficiency and Precision of Neural Abstraction [62.046646433536104]
  Neural abstractions have been recently introduced as formal approximations of complex, nonlinear dynamical models. We employ formal inductive synthesis procedures to generate neural abstractions that result in dynamical models with these semantics.
  arXiv  Detail & Related papers  (2023-07-28T13:22:32Z)
• High Accuracy Uncertainty-Aware Interatomic Force Modeling with Equivariant Bayesian Neural Networks [3.028098724882708]
  We introduce a new Monte Carlo Markov chain sampling algorithm for learning interatomic forces. In addition, we introduce a new neural network model based on the NequIP architecture and demonstrate that, when combined with our novel sampling algorithm, we obtain predictions with state-of-the-art accuracy as well as a good measure of uncertainty.
  arXiv  Detail & Related papers  (2023-04-05T10:39:38Z)
• Neural Abstractions [72.42530499990028]
  We present a novel method for the safety verification of nonlinear dynamical models that uses neural networks to represent abstractions of their dynamics. We demonstrate that our approach performs comparably to the mature tool Flow* on existing benchmark nonlinear models.
  arXiv  Detail & Related papers  (2023-01-27T12:38:09Z)
• Generalized Neural Closure Models with Interpretability [28.269731698116257]
  We develop a novel and versatile methodology of unified neural partial delay differential equations. We augment existing/low-fidelity dynamical models directly in their partial differential equation (PDE) forms with both Markovian and non-Markovian neural network (NN) closure parameterizations. We demonstrate the new generalized neural closure models (gnCMs) framework using four sets of experiments based on advecting nonlinear waves, shocks, and ocean acidification models.
  arXiv  Detail & Related papers  (2023-01-15T21:57:43Z)
• On the Forward Invariance of Neural ODEs [92.07281135902922]
  We propose a new method to ensure neural ordinary differential equations (ODEs) satisfy output specifications. Our approach uses a class of control barrier functions to transform output specifications into constraints on the parameters and inputs of the learning system.
  arXiv  Detail & Related papers  (2022-10-10T15:18:28Z)
• EINNs: Epidemiologically-Informed Neural Networks [75.34199997857341]
  We introduce a new class of physics-informed neural networks-EINN-crafted for epidemic forecasting. We investigate how to leverage both the theoretical flexibility provided by mechanistic models as well as the data-driven expressability afforded by AI models.
  arXiv  Detail & Related papers  (2022-02-21T18:59:03Z)
• Closed-form Continuous-Depth Models [99.40335716948101]
  Continuous-depth neural models rely on advanced numerical differential equation solvers. We present a new family of models, termed Closed-form Continuous-depth (CfC) networks, that are simple to describe and at least one order of magnitude faster.
  arXiv  Detail & Related papers  (2021-06-25T22:08:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.