Pretrain on Small Synthetic Data, Scale Large for Free: Symmetry-Aware Foundation Model for Logic Rule Induction

• URL: http://arxiv.org/abs/2608.00383v1
• Date: Sat, 01 Aug 2026 01:49:25 GMT
• Title: Pretrain on Small Synthetic Data, Scale Large for Free: Symmetry-Aware Foundation Model for Logic Rule Induction
• Abstract summary: Logical rule induction seeks interpretable rules that transfer across propositional schemas.<n>Our central contribution is a canonical export that decodes a discrete rule from literal scores.<n>We restore the remaining symmetries through architecture, inference, and training.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Logical rule induction seeks interpretable rules that transfer across propositional schemas. This requires respecting symmetries: atom naming, example order, polarity flips, and label swap. Enforcing exact symmetry by construction lets one trained inducer scale beyond its training schemas. Our central contribution is a canonical export that decodes a discrete rule from literal scores. It needs no retraining and is exactly equivariant whenever those scores respect the symmetries. We instantiate it on the Neural Rule Inducer, a disjunctive-normal-form (DNF) foundation model that natively respects only example order. We restore the remaining symmetries through architecture, inference, and training. On synthetic stress tests, accuracy on the support labels stays stable at much larger schemas, and rule fidelity on fresh inputs remains above the unmodified model. On real data, accuracy improves most on larger schemas. The exported rule is exact on synthetic full-group tests and on schema-valid real-data tests. This is a mathematical property of the export rather than of a specific model, and we validate it empirically only on the NRI. Enforcing symmetry by construction turns this small-data pretrained model into a reusable, interpretable inducer that transfers to larger schemas.

Related papers

• PriFT: Prior-Support Guided Supervised Fine-Tuning [74.65198014829393]
  Supervised fine-tuning (SFT) is an efficient approach for downstream task adaptation.<n>SFT fits fixed demonstrations token by token, including targets poorly aligned with the model's pretrained distribution.<n>We propose PriFT, which derives token weights from a frozen pretrained reference to obtain a stable reweighting signal unaffected by fine-tuning.
  arXiv  Detail & Related papers  (2026-06-08T12:14:06Z)
• Aligning Network Equivariance with Data Symmetry: A Theoretical Framework and Adaptive Approach for Image Restoration [71.51826063983434]
  Image restoration is an inherently ill posed inverse problem.<n>We use non strict symmetry as a constraint to formulate the restoration inverse problem.<n>We show that the equivariance error of the optimal restoration operator is strictly bounded by the data symmetry error and the discretization mesh size.
  arXiv  Detail & Related papers  (2026-05-13T16:22:19Z)
• A Foundation Model for Zero-Shot Logical Rule Induction [0.0]
  We introduce Neural Rule Inducer (NRI), a pretrained model for zero-shot rule induction.<n>Rather than encoding literal identities, NRI represents literals using domain-agnostic statistical properties.<n>Product T-norm relaxation makes rule execution differentiable, allowing end-to-end training on prediction accuracy alone.
  arXiv  Detail & Related papers  (2026-05-06T13:45:31Z)
• Balancing Symmetry and Efficiency in Graph Flow Matching [33.80974796363154]
  We study the trade-off for graph generative models between strict equivariance and controllable symmetry modulation.<n>We show that properly modulating the symmetry signal can delay overfitting while accelerating convergence.
  arXiv  Detail & Related papers  (2026-02-20T09:17:57Z)
• Equivariant Deep Equilibrium Models for Imaging Inverse Problems [50.91616288661183]
  Equivariant imaging (EI) enables training signal reconstruction models without requiring ground truth data.<n>We show that backpropagation can be implemented modularly, simplifying training.<n>We find evidence that EI-trained DEQs approximate the proximal map of an invariant prior.
  arXiv  Detail & Related papers  (2025-11-24T00:43:54Z)
• Symmetry Discovery for Different Data Types [52.2614860099811]
  Equivariant neural networks incorporate symmetries into their architecture, achieving higher generalization performance. We propose LieSD, a method for discovering symmetries via trained neural networks which approximate the input-output mappings of the tasks. We validate the performance of LieSD on tasks with symmetries such as the two-body problem, the moment of inertia matrix prediction, and top quark tagging.
  arXiv  Detail & Related papers  (2024-10-13T13:39:39Z)
• Noether's razor: Learning Conserved Quantities [16.81984465529089]
  We parameterise symmetries as learnable conserved quantities. We then allow conserved quantities and associated symmetries to be learned directly from train data. We find that our method correctly identifies the correct conserved quantities and U($n$) and SE($n$) symmetry groups.
  arXiv  Detail & Related papers  (2024-10-10T16:29:49Z)
• Disperse-Then-Merge: Pushing the Limits of Instruction Tuning via Alignment Tax Reduction [75.25114727856861]
  Large language models (LLMs) tend to suffer from deterioration at the latter stage ofSupervised fine-tuning process. We introduce a simple disperse-then-merge framework to address the issue. Our framework outperforms various sophisticated methods such as data curation and training regularization on a series of standard knowledge and reasoning benchmarks.
  arXiv  Detail & Related papers  (2024-05-22T08:18:19Z)
• Disentanglement via Latent Quantization [60.37109712033694]
  In this work, we construct an inductive bias towards encoding to and decoding from an organized latent space. We demonstrate the broad applicability of this approach by adding it to both basic data-re (vanilla autoencoder) and latent-reconstructing (InfoGAN) generative models.
  arXiv  Detail & Related papers  (2023-05-28T06:30:29Z)
• Imbalance Trouble: Revisiting Neural-Collapse Geometry [27.21274327569783]
  We introduce Simplex-Encoded-Labels Interpolation (SELI) as an invariant characterization of the neural collapse phenomenon. We prove for the UFM with cross-entropy loss and vanishing regularization. We present experiments on synthetic and real datasets that confirm convergence to the SELI geometry.
  arXiv  Detail & Related papers  (2022-08-10T18:10:59Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.