Deep Generalised Mixed Models: a Novel Neural Network Structure for Analysing Hierarchical Data

• URL: http://arxiv.org/abs/2608.05930v2
• Date: Fri, 07 Aug 2026 08:05:51 GMT
• Title: Deep Generalised Mixed Models: a Novel Neural Network Structure for Analysing Hierarchical Data
• Abstract summary: Current procedures to analyse ESM data face various challenges.<n>Machine learning procedures can give biased results due to selection bias introduced by missingness.<n>We develop a novel neural network architecture that generalises mixed effects models to deep learning to overcome these challenges.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The experience sampling method (ESM) is a longitudinal research design where participants report their thoughts, emotional states and behaviours multiple times a day. Our work is motivated by such data collected by the GrowIt! app, which was released to investigate daily emotions among adolescents during the COVID-19 pandemic. Current procedures to analyse ESM data face various challenges. While standard statistical techniques may not scale well to a high-dimensional setting, machine learning procedures can give biased results due to selection bias introduced by missingness. In our motivating dataset, adolescents dropped out due to previous strong feelings of negative emotions. Hence, the implied missing data are of the missing-at-random type that standard machine learning procedures cannot accommodate. We develop a novel neural network architecture that generalises mixed effects models to deep learning to overcome these challenges. It allows semi-parametric and flexible modelling of data's mean and correlation structure through fixed and random effects. For estimation, we use an adaptation of variational auto-encoders and a Bayesian data augmentation algorithm. Through this approach, the model can accommodate longitudinal outcomes following generic distributions, scale well to high-dimensional settings and provide valid inference when data are missing-at-random. We applied the Deep Generalised Mixed Model to the GrowIt! study and various simulations. The results show potential for the Deep Generalised Mixed Model, yet suboptimal performance due to model instability.

Related papers

• FILLER: Feature Imputation via Latent Location Exploration and Retrieval [0.8602553195689513]
  This study proposes a feature imputation method, called FILLER, that searches the two-dimensional latent space produced by a generative model and fills the missing values with appropriate entries.<n>The generative model is trained on fully observed data to generate samples from the latent space, and FILLER uses this trained model to impute the values missing in the corrupted test samples.<n>FILLER has been evaluated on several image datasets under random and structured missingness patterns with varying levels of imputation complexities.
  arXiv  Detail & Related papers  (2026-07-25T17:10:55Z)
• Grassmannian Geometry Meets Dynamic Mode Decomposition in DMD-GEN: A New Metric for Mode Collapse in Time Series Generative Models [0.0]
  Generative models like Generative Adversarial Networks (GANs) and Variational Autoencoders (Es) often fail to capture the full diversity of their training data, leading to mode collapse.<n>We introduce a new definition of mode collapse specific to time series and propose a novel metric, DMD-GEN, to quantify its severity.
  arXiv  Detail & Related papers  (2024-12-15T19:53:17Z)
• Towards Theoretical Understandings of Self-Consuming Generative Models [56.84592466204185]
  This paper tackles the emerging challenge of training generative models within a self-consuming loop. We construct a theoretical framework to rigorously evaluate how this training procedure impacts the data distributions learned by future models. We present results for kernel density estimation, delivering nuanced insights such as the impact of mixed data training on error propagation.
  arXiv  Detail & Related papers  (2024-02-19T02:08:09Z)
• Learning to Adapt Clinical Sequences with Residual Mixture of Experts [12.881413375147996]
  We propose a Mixture-of-Experts (MoE) architecture to represent complex dynamics of all patients. The architecture consists of multiple (expert) RNN models covering patient sub-populations and refining the predictions of the base model. We show 4.1% gain on AUPRC statistics compared to a single RNN prediction.
  arXiv  Detail & Related papers  (2022-04-06T09:23:12Z)
• Mixed Effects Neural ODE: A Variational Approximation for Analyzing the Dynamics of Panel Data [50.23363975709122]
  We propose a probabilistic model called ME-NODE to incorporate (fixed + random) mixed effects for analyzing panel data. We show that our model can be derived using smooth approximations of SDEs provided by the Wong-Zakai theorem. We then derive Evidence Based Lower Bounds for ME-NODE, and develop (efficient) training algorithms.
  arXiv  Detail & Related papers  (2022-02-18T22:41:51Z)
• On the Efficacy of Adversarial Data Collection for Question Answering: Results from a Large-Scale Randomized Study [65.17429512679695]
  In adversarial data collection (ADC), a human workforce interacts with a model in real time, attempting to produce examples that elicit incorrect predictions. Despite ADC's intuitive appeal, it remains unclear when training on adversarial datasets produces more robust models.
  arXiv  Detail & Related papers  (2021-06-02T00:48:33Z)
• Synthesizing Irreproducibility in Deep Networks [2.28438857884398]
  Modern day deep networks suffer from irreproducibility (also referred to as nondeterminism or underspecification) We show that even with a single nonlinearity and for very simple data and models, irreproducibility occurs. Model complexity and the choice of nonlinearity also play significant roles in making deep models irreproducible.
  arXiv  Detail & Related papers  (2021-02-21T21:51:28Z)
• Robust Finite Mixture Regression for Heterogeneous Targets [70.19798470463378]
  We propose an FMR model that finds sample clusters and jointly models multiple incomplete mixed-type targets simultaneously. We provide non-asymptotic oracle performance bounds for our model under a high-dimensional learning framework. The results show that our model can achieve state-of-the-art performance.
  arXiv  Detail & Related papers  (2020-10-12T03:27:07Z)
• Data from Model: Extracting Data from Non-robust and Robust Models [83.60161052867534]
  This work explores the reverse process of generating data from a model, attempting to reveal the relationship between the data and the model. We repeat the process of Data to Model (DtM) and Data from Model (DfM) in sequence and explore the loss of feature mapping information. Our results show that the accuracy drop is limited even after multiple sequences of DtM and DfM, especially for robust models.
  arXiv  Detail & Related papers  (2020-07-13T05:27:48Z)
• Improving Maximum Likelihood Training for Text Generation with Density Ratio Estimation [51.091890311312085]
  We propose a new training scheme for auto-regressive sequence generative models, which is effective and stable when operating at large sample space encountered in text generation. Our method stably outperforms Maximum Likelihood Estimation and other state-of-the-art sequence generative models in terms of both quality and diversity.
  arXiv  Detail & Related papers  (2020-07-12T15:31:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.