Amortized In-Context Mixed Effect Transformer Models: A Zero-Shot Approach for Pharmacokinetics

• URL: http://arxiv.org/abs/2508.15659v2
• Date: Mon, 08 Sep 2025 08:45:08 GMT
• Title: Amortized In-Context Mixed Effect Transformer Models: A Zero-Shot Approach for Pharmacokinetics
• Authors: César Ali Ojeda Marin, Wilhelm Huisinga, Purity Kavwele, Niklas Hartung,
• Abstract summary: We present the Amortized In-Context Mixed-Effect Transformer (AICMET) model.<n>It unifies mechanistic compartmental priors with amortized in-context Bayesian inference.<n>Experiments show that AICMET attains state-of-the-art predictive accuracy and faithfully quantifies inter-patient variability.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Accurate dose-response forecasting under sparse sampling is central to precision pharmacotherapy. We present the Amortized In-Context Mixed-Effect Transformer (AICMET) model, a transformer-based latent-variable framework that unifies mechanistic compartmental priors with amortized in-context Bayesian inference. AICMET is pre-trained on hundreds of thousands of synthetic pharmacokinetic trajectories with Ornstein-Uhlenbeck priors over the parameters of compartment models, endowing the model with strong inductive biases and enabling zero-shot adaptation to new compounds. At inference time, the decoder conditions on the collective context of previously profiled trial participants, generating calibrated posterior predictions for newly enrolled patients after a few early drug concentration measurements. This capability collapses traditional model-development cycles from weeks to hours while preserving some degree of expert modelling. Experiments across public datasets show that AICMET attains state-of-the-art predictive accuracy and faithfully quantifies inter-patient variability -- outperforming both nonlinear mixed-effects baselines and recent neural ODE variants. Our results highlight the feasibility of transformer-based, population-aware neural architectures as offering a new alternative for bespoke pharmacokinetic modeling pipelines, charting a path toward truly population-aware personalized dosing regimens.

Related papers

• Suppressing Prior-Comparison Hallucinations in Radiology Report Generation via Semantically Decoupled Latent Steering [94.37535002230504]
  We develop a training-free, inference-time control framework termed Semantically Decoupled Latent Steering.<n>Our approach constructs a semantic-free intervention vector via large language model (LLM)-driven semantic decomposition.<n>We show that our approach significantly reduces the probability of historical hallucinations.
  arXiv  Detail & Related papers  (2026-02-27T04:49:01Z)
• Latent Neural-ODE for Model-Informed Precision Dosing: Overcoming Structural Assumptions in Pharmacokinetics [3.0991186209192794]
  We introduce a novel data-driven alternative based on Latent Ordinary Differential Equations (Latent ODEs) for tacrolimus AUC prediction.<n>This deep learning approach learns individualized dynamics directly from sparse clinical data.<n>Latent ODE model demonstrated superior robustness, maintaining high accuracy even when underlying biological mechanisms deviated from standard assumptions.
  arXiv  Detail & Related papers  (2026-02-03T07:30:48Z)
• Improving Minimax Estimation Rates for Contaminated Mixture of Multinomial Logistic Experts via Expert Heterogeneity [49.809923981964715]
  Contaminated mixture of experts (MoE) is motivated by transfer learning methods where a pre-trained model, acting as a frozen expert, is integrated with an adapter model, functioning as a trainable expert, in order to learn a new task.<n>In this work, we characterize uniform convergence rates for estimating parameters under challenging settings where ground-truth parameters vary with the sample size.<n>We also establish corresponding minimax lower bounds to ensure that these rates are minimax optimal.
  arXiv  Detail & Related papers  (2026-01-31T23:45:50Z)
• Pretraining Transformer-Based Models on Diffusion-Generated Synthetic Graphs for Alzheimer's Disease Prediction [0.0]
  We propose a Transformer-based diagnostic framework that combines synthetic data generation with graph representation learning and transfer learning.<n>A class-conditional denoising diffusion probabilistic model (DDPM) is trained on the real-world NACC dataset to generate a large synthetic cohort.<n> Modality-specific Graph Transformer encoders are first pretrained on this synthetic data to learn robust, class-discriminative representations.
  arXiv  Detail & Related papers  (2025-11-24T19:34:53Z)
• Diffusion with a Linguistic Compass: Steering the Generation of Clinically Plausible Future sMRI Representations for Early MCI Conversion Prediction [13.937881108738042]
  We propose a diffusion-based framework that synthesizes clinically plausible future sMRI representations directly from baseline data.<n>Experiments on ADNI and AIBL cohorts show that MCI-Diff outperforms state-of-the-art baselines.
  arXiv  Detail & Related papers  (2025-06-05T07:01:05Z)
• DiffER: Categorical Diffusion for Chemical Retrosynthesis [4.8757706070066265]
  We propose DiffER, an alternative template-free method for retrosynthesis prediction in the form of categorical diffusion.<n>We construct an ensemble of diffusion models which achieves state-of-the-art performance for top-1 accuracy and competitive performance for top-3, top-5, and top-10 accuracy.
  arXiv  Detail & Related papers  (2025-05-29T17:53:37Z)
• A Generative Framework for Causal Estimation via Importance-Weighted Diffusion Distillation [55.53426007439564]
  Estimating individualized treatment effects from observational data is a central challenge in causal inference.<n>In inverse probability weighting (IPW) is a well-established solution to this problem, but its integration into modern deep learning frameworks remains limited.<n>We propose Importance-Weighted Diffusion Distillation (IWDD), a novel generative framework that combines the pretraining of diffusion models with importance-weighted score distillation.
  arXiv  Detail & Related papers  (2025-05-16T17:00:52Z)
• Predicting gene essentiality and drug response from perturbation screens in preclinical cancer models with LEAP: Layered Ensemble of Autoencoders and Predictors [4.882734501598445]
  Existing predictive models suffer from limited, generalisability and interpretability.<n>We introduce a framework of Layered Ensemble of Autoencoders and Predictors (LEAP)<n>LEAP consistently improves prediction performances in unscreened cell lines across modelling strategies.
  arXiv  Detail & Related papers  (2025-02-21T18:12:36Z)
• Adaptive Sampling to Reduce Epistemic Uncertainty Using Prediction Interval-Generation Neural Networks [0.0]
  This paper presents an adaptive sampling approach designed to reduce epistemic uncertainty in predictive models.<n>Our primary contribution is the development of a metric that estimates potential epistemic uncertainty.<n>A batch sampling strategy based on Gaussian processes (GPs) is also proposed.<n>We test our approach on three unidimensional synthetic problems and a multi-dimensional dataset based on an agricultural field for selecting experimental fertilizer rates.
  arXiv  Detail & Related papers  (2024-12-13T21:21:47Z)
• ReAugment: Model Zoo-Guided RL for Few-Shot Time Series Augmentation and Forecasting [74.00765474305288]
  We present a pilot study on using reinforcement learning (RL) for time series data augmentation.<n>Our method, ReAugment, tackles three critical questions: which parts of the training set should be augmented, how the augmentation should be performed, and what advantages RL brings to the process.
  arXiv  Detail & Related papers  (2024-09-10T07:34:19Z)
• Permutation invariant multi-output Gaussian Processes for drug combination prediction in cancer [2.1145050293719745]
  Dose-response prediction in cancer is an active application field in machine learning. The goal is to develop accurate predictive models that can be used to guide experimental design or inform treatment decisions.
  arXiv  Detail & Related papers  (2024-06-28T18:28:38Z)
• Towards Theoretical Understandings of Self-Consuming Generative Models [56.84592466204185]
  This paper tackles the emerging challenge of training generative models within a self-consuming loop. We construct a theoretical framework to rigorously evaluate how this training procedure impacts the data distributions learned by future models. We present results for kernel density estimation, delivering nuanced insights such as the impact of mixed data training on error propagation.
  arXiv  Detail & Related papers  (2024-02-19T02:08:09Z)
• Neural variational Data Assimilation with Uncertainty Quantification using SPDE priors [28.804041716140194]
  Recent advances in the deep learning community enables to address the problem through a neural architecture a variational data assimilation framework.<n>In this work we use the theory of Partial Differential Equations (SPDE) and Gaussian Processes (GP) to estimate both space-and time covariance of the state.
  arXiv  Detail & Related papers  (2024-02-02T19:18:12Z)
• Data Attribution for Diffusion Models: Timestep-induced Bias in Influence Estimation [53.27596811146316]
  Diffusion models operate over a sequence of timesteps instead of instantaneous input-output relationships in previous contexts. We present Diffusion-TracIn that incorporates this temporal dynamics and observe that samples' loss gradient norms are highly dependent on timestep. We introduce Diffusion-ReTrac as a re-normalized adaptation that enables the retrieval of training samples more targeted to the test sample of interest.
  arXiv  Detail & Related papers  (2024-01-17T07:58:18Z)
• Bayesian prognostic covariate adjustment [59.75318183140857]
  Historical data about disease outcomes can be integrated into the analysis of clinical trials in many ways. We build on existing literature that uses prognostic scores from a predictive model to increase the efficiency of treatment effect estimates.
  arXiv  Detail & Related papers  (2020-12-24T05:19:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.