Differentiate the Evaluator, Not the Program: An Efficient Runtime Representation for Neuro-Symbolic Learning

• URL: http://arxiv.org/abs/2607.03574v1
• Date: Fri, 03 Jul 2026 19:29:35 GMT
• Title: Differentiate the Evaluator, Not the Program: An Efficient Runtime Representation for Neuro-Symbolic Learning
• Abstract summary: We present the Native Differentiable Virtual Machine (NDVM), a runtime representation that differentiates executable programs without compiling each candidate into a separate graph.<n>NDVM separates symbolic structure from differentiable numeric state: tags, symbols, environments, control remain native runtime data, while numeric payloads live in dense batched buffers with exact reverse-mode gradients recorded along the realized execution trace.<n>We realize NDVM as a native runtime with forward and gradient equivalence to the backend reference Scheme, about 60x per-lane batch amortization, near-linear multicore scaling, and two independent front ends.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: AI systems increasingly propose executable scientific models whose value depends on both their symbolic structure and their fitted continuous parameters. This makes parameter calibration the bottleneck of program-and-parameter co-search: an outer loop can generate thousands of candidate programs, but each needs an inner gradient-based optimization before it can be assessed. Staging each candidate into its own differentiable graph makes individual models fast but sacrifices the program-as-data property that keeps search fluid; interpreter-based approaches preserve programs as runtime data but pay interpreter overhead that dominates the numerical work. We present the Native Differentiable Virtual Machine (NDVM), a runtime representation that differentiates executable programs without compiling each candidate into a separate graph. NDVM separates symbolic structure from differentiable numeric state: tags, symbols, environments, and control remain native runtime data, while numeric payloads live in dense batched buffers with exact reverse-mode gradients recorded along the realized execution trace, so one evaluator walk is amortized across large populations of parameter vectors. A locked cost model of a real differentiable self-hosted Scheme interpreter motivates the design. We realize NDVM as a native runtime with forward and gradient equivalence to the reference backend, about 60x per-lane batch amortization, near-linear multicore scaling, and two independent front ends. In fixed-budget co-search over LLM-proposed programs, NDVM reaches high-quality solutions about 24x sooner in wall-clock time, suggesting runtime differentiation as a practical systems foundation for scientific discovery workflows.

Related papers

• DGA$_2$D: Directed Graph-Guided Automated Algorithm Design with Large Language Models [1.8062322656999443]
  This paper proposes a Directed Graph-Guided Automated Design framework, DGA$D.<n>It structures the open-ended program space as a directed graph, where each node represents a functional operator that can be instantiated.<n>It reduces the average normalized gap by up to 10.96 percentage points compared to state-of-the-art LLM baselines.
  arXiv  Detail & Related papers  (2026-08-01T14:59:38Z)
• Compile Once, Differentiate Everywhere: A Differentiable Meta-Circular Interpreter [0.0]
  We present a compiler that translates a self-hosting subset of Scheme into differentiable graphs for autograd backends.<n>Because the subset can compile its own evaluator, this yields differentiable meta-circular interpretation (DMCI)<n>DMCI executes programs supplied as data, while reverse-mode autodiff propagates gradients to continuous constants embedded in those programs.
  arXiv  Detail & Related papers  (2026-06-07T15:13:41Z)
• OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models [69.2503510410147]
  We present a unified autonomous driving framework built upon a pretrained VLM.<n>We demonstrate that pretrained VLM attention exhibits strong transferability beyond pure language modeling.<n>Experiments on end-to-end autonomous driving benchmarks demonstrate state-of-the-art performance.
  arXiv  Detail & Related papers  (2026-04-20T07:50:00Z)
• Multivariate Long-term Time Series Forecasting with Fourier Neural Filter [42.60778405812048]
  We introduce FNF as the backbone and DBD as architecture to provide excellent learning capabilities and optimal learning pathways for spatial-temporal modeling.<n>We show that FNF unifies local time-domain and global frequency-domain information processing within a single backbone that extends naturally to spatial modeling.
  arXiv  Detail & Related papers  (2025-06-10T18:40:20Z)
• PSDNorm: Test-Time Temporal Normalization for Deep Learning in Sleep Staging [63.05435596565677]
  We propose PSDNorm that leverages Monge mapping and temporal context to normalize feature maps in deep learning models for signals.<n> PSDNorm achieves state-of-the-art performance on unseen left-out datasets while being 4-times more data-efficient than BatchNorm.
  arXiv  Detail & Related papers  (2025-03-06T16:20:25Z)
• Tempo: Compiled Dynamic Deep Learning with Symbolic Dependence Graphs [0.2578242050187029]
  We describe Tempo, a new deep learning system that combines the dynamism of eager execution with the whole-program optimizations of graph-based compilation.<n>We show that Tempo achieves a 7$times$ speedup over JAX for Llama-3.2-3B decoding.<n>For reinforcement learning algorithms, Tempo achieves a 54$times$ speedup, with 16$times$ lower peak memory usage.
  arXiv  Detail & Related papers  (2025-01-09T18:05:33Z)
• LLVM Static Analysis for Program Characterization and Memory Reuse Profile Estimation [0.0]
  This paper presents an LLVM-based probabilistic static analysis method. It accurately predicts different program characteristics and estimates the reuse distance profile of a program. The results show that our approach can predict application characteristics accurately compared to another LLVM-based dynamic code analysis tool, Byfl.
  arXiv  Detail & Related papers  (2023-11-20T23:05:06Z)
• Performance Embeddings: A Similarity-based Approach to Automatic Performance Optimization [71.69092462147292]
  Performance embeddings enable knowledge transfer of performance tuning between applications. We demonstrate this transfer tuning approach on case studies in deep neural networks, dense and sparse linear algebra compositions, and numerical weather prediction stencils.
  arXiv  Detail & Related papers  (2023-03-14T15:51:35Z)
• Scaling Structured Inference with Randomization [64.18063627155128]
  We propose a family of dynamic programming (RDP) randomized for scaling structured models to tens of thousands of latent states. Our method is widely applicable to classical DP-based inference. It is also compatible with automatic differentiation so can be integrated with neural networks seamlessly.
  arXiv  Detail & Related papers  (2021-12-07T11:26:41Z)
• High-performance symbolic-numerics via multiple dispatch [52.77024349608834]
  Symbolics.jl is an extendable symbolic system which uses dynamic multiple dispatch to change behavior depending on the domain needs. We show that by formalizing a generic API on actions independent of implementation, we can retroactively add optimized data structures to our system. We demonstrate the ability to swap between classical term-rewriting simplifiers and e-graph-based term-rewriting simplifiers.
  arXiv  Detail & Related papers  (2021-05-09T14:22:43Z)
• Exploiting Multiple Timescales in Hierarchical Echo State Networks [0.0]
  Echo state networks (ESNs) are a powerful form of reservoir computing that only require training of linear output weights. Here we explore the timescales in hierarchical ESNs, where the reservoir is partitioned into two smaller reservoirs linked with distinct properties.
  arXiv  Detail & Related papers  (2021-01-11T22:33:17Z)
• Time-varying Graph Representation Learning via Higher-Order Skip-Gram with Negative Sampling [0.456877715768796]
  We build upon the fact that the skip-gram embedding approach implicitly performs a matrix factorization. We show that higher-order skip-gram with negative sampling is able to disentangle the role of nodes and time. We empirically evaluate our approach using time-resolved face-to-face proximity data, showing that the learned time-varying graph representations outperform state-of-the-art methods.
  arXiv  Detail & Related papers  (2020-06-25T12:04:48Z)
• Connecting the Dots: Multivariate Time Series Forecasting with Graph Neural Networks [91.65637773358347]
  We propose a general graph neural network framework designed specifically for multivariate time series data. Our approach automatically extracts the uni-directed relations among variables through a graph learning module. Our proposed model outperforms the state-of-the-art baseline methods on 3 of 4 benchmark datasets.
  arXiv  Detail & Related papers  (2020-05-24T04:02:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.