Knowledge, Rules and Their Embeddings: Two Paths towards Neuro-Symbolic JEPA

• URL: http://arxiv.org/abs/2603.13265v1
• Date: Wed, 25 Feb 2026 22:31:46 GMT
• Title: Knowledge, Rules and Their Embeddings: Two Paths towards Neuro-Symbolic JEPA
• Abstract summary: We propose a bidirectional neuro-symbolic framework centered around Rule-informed Joint-Embedding Predictive Architectures (RiJEPA)<n>In the first direction, we inject structured inductive biases into JEPA training via Energy-Based Constraints (EBC) and a multi-modal dual-encoder architecture.<n>In the second direction, we demonstrate that by relaxing rigid, discrete symbolic rules into a continuous, differentiable logic, we can bypass traditional search for new rule generation.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern self-supervised predictive architectures excel at capturing complex statistical correlations from high-dimensional data but lack mechanisms to internalize verifiable human logic, leaving them susceptible to spurious correlations and shortcut learning. Conversely, traditional rule-based inference systems offer rigorous, interpretable logic but suffer from discrete boundaries and NP-hard combinatorial explosion. To bridge this divide, we propose a bidirectional neuro-symbolic framework centered around Rule-informed Joint-Embedding Predictive Architectures (RiJEPA). In the first direction, we inject structured inductive biases into JEPA training via Energy-Based Constraints (EBC) and a multi-modal dual-encoder architecture. This fundamentally reshapes the representation manifold, replacing arbitrary statistical correlations with geometrically sound logical basins. In the second direction, we demonstrate that by relaxing rigid, discrete symbolic rules into a continuous, differentiable logic, we can bypass traditional combinatorial search for new rule generation. By leveraging gradient-guided Langevin diffusion within the rule energy landscape, we introduce novel paradigms for continuous rule discovery, which enable unconditional joint generation, conditional forward and abductive inference, and marginal predictive translation. Empirical evaluations on both synthetic topological simulations and a high-stakes clinical use case confirm the efficacy of our approach. Ultimately, this framework establishes a powerful foundation for robust, generative, and interpretable neuro-symbolic representation learning.

Related papers

• Beyond the Aggregation Dilemma: Prior-Retaining Decoupled Learning for Multimodal Graphs [69.56407708794845]
  MultimodalAttributed Graph Learning (MAGL) integrates node attributes with structural topology via graph aggregation.<n>As pretrained encoders evolve into Large Foundation Models (LFMs), the landscape of MAGL fundamentally shifts.<n>Under high-confidence LFM priors, mandatory aggregation introduces topological intrinsic noise that overwhelms discriminative signals.<n>We propose SUPRA (Shared-Unique Prior-Retaining Architecture), a decoupled dual-pathway paradigm.<n> SUPRA processes modality-specific features through topology-agnostics while capturing structural synergy via a lightweight shared GNN, with auxiliary deep supervision counteracting gradient starvation.
  arXiv  Detail & Related papers  (2026-05-23T17:42:25Z)
• Information as Maximum-Caliber Deviation: A bridge between Integrated Information Theory and the Free Energy Principle [51.56484100374058]
  This work proposes that information can be defined as the deviation $$ of realized dynamics from a constrained maximum-caliber (MaxCal) path ensemble over a finite time horizon.<n>Under this definition, each of the cause/effect repertoires central to IIT 3.0 emerge directly from MaxCal variational principles.<n>Information $$ is shown to be equivalent to prediction error under accompanying predictive coding models.
  arXiv  Detail & Related papers  (2026-05-03T07:22:55Z)
• On the Rate of Convergence of GD in Non-linear Neural Networks: An Adversarial Robustness Perspective [2.268525139011456]
  We study the convergence dynamics of Gradient Descent (GD) in a minimal binary classification setting.<n>We prove that while GD successfully converges to an optimal robustness margin, this convergence occurs at a prohibitively slow rate.<n>Our theoretical guarantees are derived via a rigorous analysis of the GD trajectories across the distinct activation patterns of the model.
  arXiv  Detail & Related papers  (2026-03-02T17:13:33Z)
• KANFIS: A Neuro-Symbolic Framework for Interpretable and Uncertainty-Aware Learning [8.305093373349935]
  Adaptive Neuro-Fuzzy Inference System (ANFIS) was designed to combine the learning capabilities of neural network with the reasoning transparency of fuzzy logic.<n>We propose a compact neuro-symbolic architecture that unifies fuzzy reasoning with additive function decomposition.
  arXiv  Detail & Related papers  (2026-02-03T03:02:17Z)
• Bidirectional Representations Augmented Autoregressive Biological Sequence Generation:Application in De Novo Peptide Sequencing [51.12821379640881]
  Autoregressive (AR) models offer holistic, bidirectional representations but face challenges with generative coherence and scalability.<n>We propose a hybrid framework enhancing AR generation by dynamically integrating rich contextual information from non-autoregressive mechanisms.<n>A novel cross-decoder attention module enables the AR decoder to iteratively query and integrate these bidirectional features.
  arXiv  Detail & Related papers  (2025-10-09T12:52:55Z)
• Understanding Transformers through the Lens of Pavlovian Conditioning [0.5076419064097734]
  We present a novel theoretical framework that reinterprets the core computation of attention as Pavlovian conditioning.<n>We demonstrate that attention's queries, keys, and values can be mapped to the three elements of classical conditioning.<n>Our framework yields several theoretical insights grounded in this linearized model.
  arXiv  Detail & Related papers  (2025-08-05T05:00:00Z)
• Why Neural Network Can Discover Symbolic Structures with Gradient-based Training: An Algebraic and Geometric Foundation for Neurosymbolic Reasoning [73.18052192964349]
  We develop a theoretical framework that explains how discrete symbolic structures can emerge naturally from continuous neural network training dynamics.<n>By lifting neural parameters to a measure space and modeling training as Wasserstein gradient flow, we show that under geometric constraints, the parameter measure $mu_t$ undergoes two concurrent phenomena.
  arXiv  Detail & Related papers  (2025-06-26T22:40:30Z)
• Certified Neural Approximations of Nonlinear Dynamics [51.01318247729693]
  In safety-critical contexts, the use of neural approximations requires formal bounds on their closeness to the underlying system.<n>We propose a novel, adaptive, and parallelizable verification method based on certified first-order models.
  arXiv  Detail & Related papers  (2025-05-21T13:22:20Z)
• Modeling Arbitrarily Applicable Relational Responding with the Non-Axiomatic Reasoning System: A Machine Psychology Approach [0.0]
  We present a novel theoretical approach for modeling AARR within an artificial intelligence framework using the Non-Axiomatic Reasoning System (NARS)<n>We show how key properties of AARR can emerge from the inference rules and memory structures of NARS.<n>Results suggest that AARR can be conceptually captured by suitably designed AI systems.
  arXiv  Detail & Related papers  (2025-03-01T20:37:11Z)
• A Novel Neural-symbolic System under Statistical Relational Learning [47.30190559449236]
  We propose a neural-symbolic framework based on statistical relational learning, referred to as NSF-SRL.<n>Results of symbolic reasoning are utilized to refine and correct the predictions made by deep learning models, while deep learning models enhance the efficiency of the symbolic reasoning process.<n>We believe that this approach sets a new standard for neural-symbolic systems and will drive future research in the field of general artificial intelligence.
  arXiv  Detail & Related papers  (2023-09-16T09:15:37Z)
• Relational Reasoning Networks [9.262082274025708]
  This paper presents Neuro-symbolic Networks (R2N), a novel end-to-end model that performs reasoning in the latent space of a deep architecture.<n>R2Ns can be applied to purely symbolic or tasks as a neuro-symbolic platform to integrate learning and reasoning in heterogeneous problems.
  arXiv  Detail & Related papers  (2021-06-01T11:02:22Z)
• An Ode to an ODE [78.97367880223254]
  We present a new paradigm for Neural ODE algorithms, called ODEtoODE, where time-dependent parameters of the main flow evolve according to a matrix flow on the group O(d) This nested system of two flows provides stability and effectiveness of training and provably solves the gradient vanishing-explosion problem.
  arXiv  Detail & Related papers  (2020-06-19T22:05:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.