From Specification to Architecture: A Theory Compiler for Knowledge-Guided Machine Learning

• URL: http://arxiv.org/abs/2603.14369v1
• Date: Sun, 15 Mar 2026 13:15:31 GMT
• Title: From Specification to Architecture: A Theory Compiler for Knowledge-Guided Machine Learning
• Abstract summary: Theory-guided machine learning has demonstrated that including authentic domain knowledge directly into model design improves performance.<n>Yet the process by which a formal domain theory is translated into architectural constraints remains entirely manual.<n>We propose the Theory Compiler: a system that accepts a typed, machine-readable domain theory as input and automatically produces an architecture.
• Score: 6.365277569082624
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Theory-guided machine learning has demonstrated that including authentic domain knowledge directly into model design improves performance, sample efficiency and out-of-distribution generalisation. Yet the process by which a formal domain theory is translated into architectural constraints remains entirely manual, specific to each domain formalism, and devoid of any formal correctness guarantee. This translation is non-transferable between domains, not verified, and does not scale. We propose the Theory Compiler: a system that accepts a typed, machine-readable domain theory as input and automatically produces an architecture whose function space is provably constrained to be consistent with that theory by construction, not by regularisation. We identify three foundational open problems whose resolution defines our research agenda: (1) designing a universal theory formalisation language with decidable type-checking; (2) constructing a compositionally correct compilation algorithm from theory primitives to architectural modules; and (3) establishing soundness and completeness criteria for formal verification. We further conjecture that compiled architectures match or exceed manually-designed counterparts in generalisation performance while requiring substantially less training data, a claim we ground in classical statistical learning theory. We argue that recent advances in formal machine learning theory, large language models, and the growth of an interdisciplinary research community have made this paradigm achievable for the first time.

Related papers

• Beyond $L_2$: Generalizing Abductive Latent Explanations to Diverse Prototype-Based Architectures [7.001970497421476]
  Abductive Latent Explanations (ALE) were introduced to provide formal, mathematically guaranteed explanations.<n>ALEs rely on computing tight bounds on latent space distances to produce formal explanations.<n>In this work, we generalize the ALE framework to support non-Euclidean prototype architectures.
  arXiv  Detail & Related papers  (2026-08-17T16:20:20Z)
• A Limit Theory of Foundation Models: A Mathematical Approach to Understanding Emergent Intelligence and Scaling Laws [69.11818285240689]
  This study attempts to develop a mathematical approach to formalize emergent intelligence from the perspective of limit theory.<n>We recast emergent intelligence as existence of the limit $lim_N,P,K to infty mathcalE(N,P,K)$, with emergent abilities corresponding to the limiting behavior.
  arXiv  Detail & Related papers  (2026-04-27T04:43:42Z)
• Decidable By Construction: Design-Time Verification for Trustworthy AI [0.0]
  A prevailing assumption in machine learning is that model correctness must be enforced after the fact.<n>We observe that the properties determining whether an AI model is numerically stable, computationally correct, or consistent with a physical domain do not necessarily demand post hoc enforcement.
  arXiv  Detail & Related papers  (2026-03-26T13:09:36Z)
• Adapting, Fast and Slow: Transportable Circuits for Few-Shot Learning [54.930879235929204]
  Generalization across the domains is not possible without asserting a structure that constrains the unseen target domain w.r.t.<n>We design an algorithm for zero-shot compositional generalization which relies on access to qualitative domain knowledge.<n>Our theoretical results characterize classes of few-shot learnable tasks in terms of graphical circuit transportability criteria.
  arXiv  Detail & Related papers  (2025-12-28T04:38:43Z)
• BRIDGE: Building Representations In Domain Guided Program Verification [67.36686119518441]
  BRIDGE decomposes verification into three interconnected domains: Code, Specifications, and Proofs.<n>We show that this approach substantially improves both accuracy and efficiency beyond standard error feedback methods.
  arXiv  Detail & Related papers  (2025-11-26T06:39:19Z)
• Emergent Cognitive Convergence via Implementation: A Structured Loop Reflecting Four Theories of Mind [0.0]
  We report a structural convergence among four influential theories of mind.<n>The convergence emerges unintentionally within a practical AI architecture known as Agentic Flow.<n>This paper proposes that intelligent architectures may evolve toward shared structural patterns shaped by practical constraints.
  arXiv  Detail & Related papers  (2025-07-22T02:54:45Z)
• Scaling Laws and Representation Learning in Simple Hierarchical Languages: Transformers vs. Convolutional Architectures [49.19753720526998]
  We derive theoretical scaling laws for neural network performance on synthetic datasets.<n>We validate that convolutional networks, whose structure aligns with that of the generative process through locality and weight sharing, enjoy a faster scaling of performance.<n>This finding clarifies the architectural biases underlying neural scaling laws and highlights how representation learning is shaped by the interaction between model architecture and the statistical properties of data.
  arXiv  Detail & Related papers  (2025-05-11T17:44:14Z)
• Categorical semiotics: Foundations for Knowledge Integration [0.0]
  We tackle the challenging task of developing a comprehensive framework for defining and analyzing deep learning architectures. Our methodology employs graphical structures that resemble Ehresmann's sketches, interpreted within a universe of fuzzy sets. This approach offers a unified theory that elegantly encompasses both deterministic and non-deterministic neural network designs.
  arXiv  Detail & Related papers  (2024-04-01T23:19:01Z)
• Hierarchical Invariance for Robust and Interpretable Vision Tasks at Larger Scales [54.78115855552886]
  We show how to construct over-complete invariants with a Convolutional Neural Networks (CNN)-like hierarchical architecture. With the over-completeness, discriminative features w.r.t. the task can be adaptively formed in a Neural Architecture Search (NAS)-like manner. For robust and interpretable vision tasks at larger scales, hierarchical invariant representation can be considered as an effective alternative to traditional CNN and invariants.
  arXiv  Detail & Related papers  (2024-02-23T16:50:07Z)
• Position: Categorical Deep Learning is an Algebraic Theory of All Architectures [1.772996392520906]
  We present our position on the quest for a general-purpose framework for specifying and studying deep learning architectures. We propose to apply category theory as a single theory elegantly subsuming both of these flavours of neural network design. We show how this theory recovers constraints induced by geometric deep learning, as well as implementations of many architectures drawn from the diverse landscape of neural networks.
  arXiv  Detail & Related papers  (2024-02-23T14:01:53Z)
• A Theory of Label Propagation for Subpopulation Shift [61.408438422417326]
  We propose a provably effective framework for domain adaptation based on label propagation. We obtain end-to-end finite-sample guarantees on the entire algorithm. We extend our theoretical framework to a more general setting of source-to-target transfer based on a third unlabeled dataset.
  arXiv  Detail & Related papers  (2021-02-22T17:27:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.