Neuro-Symbolic Activation Discovery: Transferring Mathematical Structures from Physics to Ecology for Parameter-Efficient Neural Networks

• URL: http://arxiv.org/abs/2601.10740v1
• Date: Sat, 10 Jan 2026 02:49:32 GMT
• Title: Neuro-Symbolic Activation Discovery: Transferring Mathematical Structures from Physics to Ecology for Parameter-Efficient Neural Networks
• Abstract summary: We propose Neuro-Symbolic Activation Discovery, a framework that uses Genetic Programming to extract interpretable mathematical formulas from data.<n>Our key contribution is the discovery of a Geometric Transfer phenomenon: activation functions learned from particle physics data successfully generalize to ecological classification.<n>Our work opens pathways toward domain-specific activation libraries for efficient scientific machine learning.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Modern neural networks rely on generic activation functions (ReLU, GELU, SiLU) that ignore the mathematical structure inherent in scientific data. We propose Neuro-Symbolic Activation Discovery, a framework that uses Genetic Programming to extract interpretable mathematical formulas from data and inject them as custom activation functions. Our key contribution is the discovery of a Geometric Transfer phenomenon: activation functions learned from particle physics data successfully generalize to ecological classification, outperforming standard activations (ReLU, GELU, SiLU) in both accuracy and parameter efficiency. On the Forest Cover dataset, our Hybrid Transfer model achieves 82.4% accuracy with only 5,825 parameters, compared to 83.4% accuracy requiring 31,801 parameters for a conventional heavy network -- a 5.5x parameter reduction with only 1% accuracy loss. We introduce a Parameter Efficiency Score ($E_{param} = AUC / \log_{10}(Params)$) and demonstrate that lightweight hybrid architectures consistently achieve 18-21% higher efficiency than over-parameterized baselines. Crucially, we establish boundary conditions: while Physics to Ecology transfer succeeds (both involve continuous Euclidean measurements), Physics to Text transfer fails (discrete word frequencies require different mathematical structures). Our work opens pathways toward domain-specific activation libraries for efficient scientific machine learning.

Related papers

• Neuron-Guided Fine-Tuning: Unlocking Efficient Alignment Mechanisms for Large Language Models [73.88918843965155]
  Existing Supervised Fine-Tuning paradigms are often plagued by parameter redundancy, inconsistent data quality, and catastrophic forgetting.<n>We propose Neuron-Guided Fine-Tuning (NGFT), a holistic framework that leverages neuron activation patterns as a universal proxy to unify the fine-tuning lifecycle.
  arXiv  Detail & Related papers  (2026-09-05T06:13:24Z)
• Parameter-Efficient Conditioning for Material Generalization in Graph-Based Simulators [2.504298819189614]
  Graph network-based simulators (GNS) have demonstrated strong potential for learning particle-based physics.<n>Existing models are typically trained for a single material type and fail to generalize across distinct behaviors.<n>We propose a parameter-efficient conditioning mechanism that makes the GNS model adaptive to material parameters.
  arXiv  Detail & Related papers  (2025-11-07T17:55:35Z)
• MaNGO - Adaptable Graph Network Simulators via Meta-Learning [25.80703650406406]
  Graph Network Simulators (GNSs) offer faster inference but suffer from two key limitations.<n>They must be retrained from scratch for even minor variations in physical parameters.<n>This is inefficient, as simulations with varying parameters often share a common underlying latent structure.<n>We propose a novel architecture that generates a latent representation by encoding graph trajectories using conditional neural processes.
  arXiv  Detail & Related papers  (2025-10-07T12:44:24Z)
• Towards Understanding Feature Learning in Parameter Transfer [47.063219231351916]
  We analyze a setting in which both the upstream and downstream models are ReLU convolutional neural networks (CNNs)<n>We characterize how the inherited parameters act as carriers of universal knowledge and identify key factors that amplify their beneficial impact on the target task.<n>Our analysis provides insight into why, in certain cases, transferring parameters can lead to lower test accuracy on the target task than training a new model from scratch.
  arXiv  Detail & Related papers  (2025-09-26T08:37:54Z)
• Developing Training Procedures for Piecewise-linear Spline Activation Functions in Neural Networks [0.0]
  I present and compare 9 training methodologies to explore dual-optimization dynamics in neural networks.<n>Experiments realize up to 94% lower end model error rates in FNNs and 51% lower rates in CNNs compared to traditional ReLU-based models.
  arXiv  Detail & Related papers  (2025-09-17T03:51:16Z)
• Global Convergence and Rich Feature Learning in $L$-Layer Infinite-Width Neural Networks under $μ$P Parametrization [66.03821840425539]
  In this paper, we investigate the training dynamics of $L$-layer neural networks using the tensor gradient program (SGD) framework.<n>We show that SGD enables these networks to learn linearly independent features that substantially deviate from their initial values.<n>This rich feature space captures relevant data information and ensures that any convergent point of the training process is a global minimum.
  arXiv  Detail & Related papers  (2025-03-12T17:33:13Z)
• Learning Hidden Physics and System Parameters with Deep Operator Networks [0.0]
  We introduce two innovative neural operator frameworks tailored for discovering hidden physics and identifying unknown system parameters from sparse measurements.<n>The first framework integrates a popular neural operator, DeepONet, and a physics-informed neural network to capture the relationship between sparse data and the underlying physics, enabling the accurate discovery of a family of governing equations.<n>The second framework focuses on system parameter identification, leveraging a DeepONet pre-trained on sparse sensor measurements to initialize a physics-constrained inverse model.
  arXiv  Detail & Related papers  (2024-12-06T15:44:59Z)
• Active Learning with Fully Bayesian Neural Networks for Discontinuous and Nonstationary Data [0.0]
  We introduce fully Bayesian Neural Networks (FBNNs) for active learning tasks in the'small data' regime. FBNNs provide reliable predictive distributions, crucial for making informed decisions under uncertainty in the active learning setting. Here, we assess the suitability and performance of FBNNs with the No-U-Turn Sampler for active learning tasks in the'small data' regime.
  arXiv  Detail & Related papers  (2024-05-16T05:20:47Z)
• Gradual Optimization Learning for Conformational Energy Minimization [69.36925478047682]
  Gradual Optimization Learning Framework (GOLF) for energy minimization with neural networks significantly reduces the required additional data. Our results demonstrate that the neural network trained with GOLF performs on par with the oracle on a benchmark of diverse drug-like molecules.
  arXiv  Detail & Related papers  (2023-11-05T11:48:08Z)
• 19 Parameters Is All You Need: Tiny Neural Networks for Particle Physics [52.42485649300583]
  We present the potential of one recent Lorentz- and permutation-symmetric architecture, PELICAN, for low-latency neural network tasks. We show its instances with as few as 19 trainable parameters that outperform generic architectures with tens of thousands of parameters when compared on the binary classification task of top quark jet tagging.
  arXiv  Detail & Related papers  (2023-10-24T18:51:22Z)
• Globally Optimal Training of Neural Networks with Threshold Activation Functions [63.03759813952481]
  We study weight decay regularized training problems of deep neural networks with threshold activations. We derive a simplified convex optimization formulation when the dataset can be shattered at a certain layer of the network.
  arXiv  Detail & Related papers  (2023-03-06T18:59:13Z)
• A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networks [65.34977803841007]
  Predictive coding networks are neuroscience-inspired models with roots in both Bayesian statistics and neuroscience. We show how by simply changing the temporal scheduling of the update rule for the synaptic weights leads to an algorithm that is much more efficient and stable than the original one.
  arXiv  Detail & Related papers  (2022-11-16T00:11:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.