Compile Once, Differentiate Everywhere: A Differentiable Meta-Circular Interpreter

• URL: http://arxiv.org/abs/2606.09930v1
• Date: Sun, 07 Jun 2026 15:13:41 GMT
• Title: Compile Once, Differentiate Everywhere: A Differentiable Meta-Circular Interpreter
• Abstract summary: We present a compiler that translates a self-hosting subset of Scheme into differentiable graphs for autograd backends.<n>Because the subset can compile its own evaluator, this yields differentiable meta-circular interpretation (DMCI)<n>DMCI executes programs supplied as data, while reverse-mode autodiff propagates gradients to continuous constants embedded in those programs.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The boundary between program execution and gradient-based optimization has long limited the use of code itself as a learnable scientific model. We present a compiler that translates a self-hosting subset of Scheme into differentiable computation graphs for autograd backends. Because the subset can compile its own evaluator, this yields differentiable meta-circular interpretation (DMCI): a compiled Scheme interpreter executes programs supplied as data, while reverse-mode autodiff propagates gradients to continuous constants embedded in those programs. The interpreter is compiled once, so new programs inherit differentiability without recompilation or custom gradient machinery, while retaining closures, recursion, and data structures. We prove that gradients through the compiled interpreter are correct almost everywhere and show that they match direct compilation to numerical precision across 171 recursive and higher-order program-seed pairs. We then use DMCI for program-and-parameter co-search, where a large language model proposes Scheme programs and exact gradients calibrate their continuous parameters through a single frozen interpreter. This enables OpenEvolve-style program search in which an outer loop proposes discrete program structures and DMCI supplies exact gradient-based calibration of each candidate's continuous parameters. On battery capacity-fade data, the search recovers a knee-like degradation structure and improves held-out extrapolation over hand-crafted baselines on the harder early-extrapolation split, matching them on the later split. On a high-dimensional El Nino inverse problem, DMCI optimizes an interpreted Kalman-filter likelihood where gradient-free search fails. These results extend symbolic regression and neurosymbolic search from closed-form expressions to executable, stateful programs, making model-generated code directly optimizable against data.

Related papers

• Overcoming the Weakest-Link Effect in LLM-Driven Program Optimization via Heterogeneous Edit Recombination [59.679182823718556]
  HERO is a program that generates diverse, non-overlapping atomic edits and then systematically selects and composes them into coherent program improvements using evaluator scores.<n>We evaluate HERO across algorithmic problems, strategy games, the design of LLM-based agentic systems, and robotic path planning.
  arXiv  Detail & Related papers  (2026-07-31T02:05:45Z)
• Kaizen: Metamorphic Fuzzing and Differential Testing for LLM-Translated HPC Applications [2.4948485991003158]
  We present a framework for evaluating correctness of large language models (LLMs) translated to OpenMP, OpenACC, Kokkos or SYCL.<n>We evaluate on-to-OpenMP translation of 16 scientific applications using three fine-tuned LLMs at kernel-level and full-program granularity.
  arXiv  Detail & Related papers  (2026-07-04T23:52:39Z)
• Differentiate the Evaluator, Not the Program: An Efficient Runtime Representation for Neuro-Symbolic Learning [0.0]
  We present the Native Differentiable Virtual Machine (NDVM), a runtime representation that differentiates executable programs without compiling each candidate into a separate graph.<n>NDVM separates symbolic structure from differentiable numeric state: tags, symbols, environments, control remain native runtime data, while numeric payloads live in dense batched buffers with exact reverse-mode gradients recorded along the realized execution trace.<n>We realize NDVM as a native runtime with forward and gradient equivalence to the backend reference Scheme, about 60x per-lane batch amortization, near-linear multicore scaling, and two independent front ends.
  arXiv  Detail & Related papers  (2026-07-03T19:29:35Z)
• Latent Heuristic Search: Continuous Optimization for Automated Algorithm Design [6.870020562464171]
  This work proposes a continuous discovery framework that navigates a non-competitive landscape.<n>We use an embedding gradient map algorithm into continuous embeddings of latent differentiable surrogates.<n>The resulting optimized executables are projected through a learned mapper into soft prompts.
  arXiv  Detail & Related papers  (2026-05-16T20:03:01Z)
• HyCOP: Hybrid Composition Operators for Interpretable Learning of PDEs [6.796343285359228]
  HyCOP learns PDE solution operators by composing simple modules in a query-conditioned way.<n>Rather than learning a monolithic map, HyCOP learns a policy over short programs conditioned on regime features and state statistics.
  arXiv  Detail & Related papers  (2026-05-01T17:57:48Z)
• Superset Decompilation [7.989066784325648]
  decompilation is the reverse of compilation and can be structured as a sequence of modular passes.<n>We formalize this as provenance-guided superset decompilation (PGSD), a framework that monotonically derives facts about the binary into a relation store.<n>Mandel implements PGSD as a declarative reverse engineering framework that lifts Linux ELF binaries to C99 through a granular intermediate representation in 35K lines of Rust and Datalog.
  arXiv  Detail & Related papers  (2026-03-30T03:47:19Z)
• INC: An Indirect Neural Corrector for Auto-Regressive Hybrid PDE Solvers [61.84396402100827]
  We propose the Indirect Neural Corrector ($mathrmINC$), which integrates learned corrections into the governing equations.<n>$mathrmINC$ reduces the error amplification on the order of $t-1 + L$, where $t$ is the timestep and $L$ the Lipschitz constant.<n>We test $mathrmINC$ in extensive benchmarks, covering numerous differentiable solvers, neural backbones, and test cases ranging from a 1D chaotic system to 3D turbulence.
  arXiv  Detail & Related papers  (2025-11-16T20:14:28Z)
• A Compiler from Array Programs to Vectorized Homomorphic Encryption [1.6216324006136673]
  Homomorphic encryption (HE) is a practical approach to secure computation over encrypted data. We present Viaduct-HE, a compiler generates efficient vectorized HE programs. Viaduct-HE can generate both the operations and complex data layouts required for efficient HE programs.
  arXiv  Detail & Related papers  (2023-11-10T16:00:00Z)
• Performance Embeddings: A Similarity-based Approach to Automatic Performance Optimization [71.69092462147292]
  Performance embeddings enable knowledge transfer of performance tuning between applications. We demonstrate this transfer tuning approach on case studies in deep neural networks, dense and sparse linear algebra compositions, and numerical weather prediction stencils.
  arXiv  Detail & Related papers  (2023-03-14T15:51:35Z)
• Source-to-Source Automatic Differentiation of OpenMP Parallel Loops [0.0]
  This paper presents our work toward correct and efficient automatic differentiation of OpenMP parallel worksharing loops in forward and reverse mode. We propose a framework to reason about the correctness of the generated derivative code, from which we justify our OpenMP extension to the differentiation model. Performance of the generated derivative programs in forward and reverse mode is better than sequential, although our reverse mode often scales worse than the input programs.
  arXiv  Detail & Related papers  (2021-11-02T19:40:59Z)
• Unfolding Projection-free SDP Relaxation of Binary Graph Classifier via GDPA Linearization [59.87663954467815]
  Algorithm unfolding creates an interpretable and parsimonious neural network architecture by implementing each iteration of a model-based algorithm as a neural layer. In this paper, leveraging a recent linear algebraic theorem called Gershgorin disc perfect alignment (GDPA), we unroll a projection-free algorithm for semi-definite programming relaxation (SDR) of a binary graph. Experimental results show that our unrolled network outperformed pure model-based graph classifiers, and achieved comparable performance to pure data-driven networks but using far fewer parameters.
  arXiv  Detail & Related papers  (2021-09-10T07:01:15Z)
• Efficient Learning of Generative Models via Finite-Difference Score Matching [111.55998083406134]
  We present a generic strategy to efficiently approximate any-order directional derivative with finite difference. Our approximation only involves function evaluations, which can be executed in parallel, and no gradient computations.
  arXiv  Detail & Related papers  (2020-07-07T10:05:01Z)
• Predictive Coding Approximates Backprop along Arbitrary Computation Graphs [68.8204255655161]
  We develop a strategy to translate core machine learning architectures into their predictive coding equivalents. Our models perform equivalently to backprop on challenging machine learning benchmarks. Our method raises the potential that standard machine learning algorithms could in principle be directly implemented in neural circuitry.
  arXiv  Detail & Related papers  (2020-06-07T15:35:47Z)
• Variance Reduction with Sparse Gradients [82.41780420431205]
  Variance reduction methods such as SVRG and SpiderBoost use a mixture of large and small batch gradients. We introduce a new sparsity operator: The random-top-k operator. Our algorithm consistently outperforms SpiderBoost on various tasks including image classification, natural language processing, and sparse matrix factorization.
  arXiv  Detail & Related papers  (2020-01-27T08:23:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.