Implementing neural network mixed-effects models in Template Model Builder (TMB)

• URL: http://arxiv.org/abs/2608.31133v1
• Date: Mon, 31 Aug 2026 17:43:02 GMT
• Title: Implementing neural network mixed-effects models in Template Model Builder (TMB)
• Abstract summary: We introduce a general framework for implementing Neural network mixed-effects models (NMMs) using template model builder (TMB)<n>TMB requires users to specify only the negative joint log-likelihood and any regularization terms.<n>We demonstrate the efficiency, flexibility, and statistical performance of TMB-based NMMs across two numerical examples.
• Score: 1.1352560842946413
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Neural network mixed-effects models (NMMs) have gained traction by combining the strong representation and predictive power of artificial neural networks with the capacity of mixed-effects modeling to capture complex correlation structures. However, existing estimation approaches rely heavily on manual derivations of objective functions and gradients, which inherently forces simplifying approximations and severely constrains the complexity and accuracy of NMMs. In this work, we introduce a general framework for implementing NMMs using Template Model Builder (TMB). By leveraging automatic differentiation and Laplace approximation, TMB requires users to specify only the negative joint log-likelihood and any regularization terms. The framework automatically integrates out random effects and evaluates the marginal objective function alongside its exact gradients, eliminating the need for manual derivations or ad hoc approximations. We demonstrate the efficiency, flexibility, and statistical performance of TMB-based NMMs across two numerical examples, including an application to monotonic NMMs. Reproducible code is provided to facilitate broader adoption.

Related papers

• Mixture-of-Models: Unifying Heterogeneous Agents via N-Way Self-Evaluating Deliberation [0.0]
  This paper introduces the N-Way Self-Evaluating Deliberation (NSED) protocol, a Mixture-of-Models (MoM) architecture that constructs emergent composite models from a plurality of distinct expert agents.<n>Unlike traditional Mixture-of-Experts (MoE) which rely on static gating networks, NSED employs a Dynamic Expertise Broker that treats model selection as a variation of the Knapsack Problem.
  arXiv  Detail & Related papers  (2026-01-23T16:11:54Z)
• uGMM-NN: Univariate Gaussian Mixture Model Neural Network [0.0]
  uGMM-NN is a novel neural architecture that embeds probabilistic reasoning directly into the computational units of deep networks.<n>We demonstrate that uGMM-NN can achieve competitive discriminative performance compared to conventional multilayer perceptrons.
  arXiv  Detail & Related papers  (2025-09-09T10:13:37Z)
• Towards Efficient General Feature Prediction in Masked Skeleton Modeling [59.46799426434277]
  We propose a novel General Feature Prediction framework (GFP) for efficient mask skeleton modeling.<n>Our key innovation is replacing conventional low-level reconstruction with high-level feature prediction that spans from local motion patterns to global semantic representations.
  arXiv  Detail & Related papers  (2025-09-03T18:05:02Z)
• NAN: A Training-Free Solution to Coefficient Estimation in Model Merging [61.36020737229637]
  We show that the optimal merging weights should scale with the amount of task-specific information encoded in each model.<n>We propose NAN, a simple yet effective method that estimates model merging coefficients via the inverse of parameter norm.<n>NAN is training-free, plug-and-play, and applicable to a wide range of merging strategies.
  arXiv  Detail & Related papers  (2025-05-22T02:46:08Z)
• Reinforced Model Merging [53.84354455400038]
  We present an innovative framework termed Reinforced Model Merging (RMM), which encompasses an environment and agent tailored for merging tasks.<n>By utilizing data subsets during the evaluation process, we addressed the bottleneck in the reward feedback phase, thereby accelerating RMM by up to 100 times.
  arXiv  Detail & Related papers  (2025-03-27T08:52:41Z)
• ALWNN Empowered Automatic Modulation Classification: Conquering Complexity and Scarce Sample Conditions [24.59462798452397]
  This paper proposes an automatic modulation classification model based on the Adaptive Lightweight Wavelet Neural Network (ALWNN) and the few-shot framework (MALWNN)<n>The ALWNN model, by integrating the adaptive wavelet neural network and depth separable convolution, reduces the number of model parameters and computational complexity.<n> Experiments with MALWNN show its superior performance in few-shot learning scenarios compared to other algorithms.
  arXiv  Detail & Related papers  (2025-03-24T06:14:33Z)
• End-to-End Reinforcement Learning of Koopman Models for Economic Nonlinear Model Predictive Control [45.84205238554709]
  We present a method for reinforcement learning of Koopman surrogate models for optimal performance as part of (e)NMPC. We show that the end-to-end trained models outperform those trained using system identification in (e)NMPC.
  arXiv  Detail & Related papers  (2023-08-03T10:21:53Z)
• Smooth Min-Max Monotonic Networks [9.313156974838076]
  Monotonicity constraints are powerful regularizers in statistical modelling. They can support fairness in computer-aided decision making and increase plausibility in data-driven scientific models. The seminal min-max (MM) neural network architecture ensures monotonicity, but often gets stuck in undesired local optima during training because of partial derivatives of the MM nonlinearities being zero. We propose a simple modification of the MM network using strictly-increasing smooth minimum and maximum functions that alleviates this problem.
  arXiv  Detail & Related papers  (2023-06-01T21:03:06Z)
• Personalized Federated Learning under Mixture of Distributions [98.25444470990107]
  We propose a novel approach to Personalized Federated Learning (PFL), which utilizes Gaussian mixture models (GMM) to fit the input data distributions across diverse clients. FedGMM possesses an additional advantage of adapting to new clients with minimal overhead, and it also enables uncertainty quantification. Empirical evaluations on synthetic and benchmark datasets demonstrate the superior performance of our method in both PFL classification and novel sample detection.
  arXiv  Detail & Related papers  (2023-05-01T20:04:46Z)
• MoEfication: Conditional Computation of Transformer Models for Efficient Inference [66.56994436947441]
  Transformer-based pre-trained language models can achieve superior performance on most NLP tasks due to large parameter capacity, but also lead to huge computation cost. We explore to accelerate large-model inference by conditional computation based on the sparse activation phenomenon. We propose to transform a large model into its mixture-of-experts (MoE) version with equal model size, namely MoEfication.
  arXiv  Detail & Related papers  (2021-10-05T02:14:38Z)
• Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learning [66.18202188565922]
  We propose a communication-efficient decentralized machine learning (ML) algorithm, coined QGADMM (QGADMM)<n>We develop a novel quantization method to adaptively adjust modelization levels and their probabilities, while proving the convergence of QGADMM for convex functions.
  arXiv  Detail & Related papers  (2019-10-23T10:47:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.