Policy Optimization in Bayesian Network Hybrid Models of Biomanufacturing Processes

• URL: http://arxiv.org/abs/2105.06543v1
• Date: Thu, 13 May 2021 20:39:02 GMT
• Title: Policy Optimization in Bayesian Network Hybrid Models of Biomanufacturing Processes
• Authors: Hua Zheng, Wei Xie, Ilya O. Ryzhov, Dongming Xie
• Abstract summary: Biomanufacturing processes require close monitoring and control. We develop a novel model-based reinforcement learning framework that can achieve human-level control in low-data environments.
• Score: 3.124775036986647
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Biopharmaceutical manufacturing is a rapidly growing industry with impact in virtually all branches of medicine. Biomanufacturing processes require close monitoring and control, in the presence of complex bioprocess dynamics with many interdependent factors, as well as extremely limited data due to the high cost and long duration of experiments. We develop a novel model-based reinforcement learning framework that can achieve human-level control in low-data environments. The model uses a probabilistic knowledge graph to capture causal interdependencies between factors in the underlying stochastic decision process, leveraging information from existing kinetic models from different unit operations while incorporating real-world experimental data. We then present a computationally efficient, provably convergent stochastic gradient method for policy optimization. Validation is conducted on a realistic application with a multi-dimensional, continuous state variable.

Related papers

• Distributionally Robust Model-based Reinforcement Learning with Large State Spaces [55.14361269378122]
  Three major challenges in reinforcement learning are the complex dynamical systems with large state spaces, the costly data acquisition processes, and the deviation of real-world dynamics from the training environment deployment. We study distributionally robust Markov decision processes with continuous state spaces under the widely used Kullback-Leibler, chi-square, and total variation uncertainty sets. We propose a model-based approach that utilizes Gaussian Processes and the maximum variance reduction algorithm to efficiently learn multi-output nominal transition dynamics.
  arXiv  Detail & Related papers  (2023-09-05T13:42:11Z)
• Learning Latent Dynamics via Invariant Decomposition and (Spatio-)Temporal Transformers [0.6767885381740952]
  We propose a method for learning dynamical systems from high-dimensional empirical data. We focus on the setting in which data are available from multiple different instances of a system. We study behaviour through simple theoretical analyses and extensive experiments on synthetic and real-world datasets.
  arXiv  Detail & Related papers  (2023-06-21T07:52:07Z)
• Latent Variable Representation for Reinforcement Learning [131.03944557979725]
  It remains unclear theoretically and empirically how latent variable models may facilitate learning, planning, and exploration to improve the sample efficiency of model-based reinforcement learning. We provide a representation view of the latent variable models for state-action value functions, which allows both tractable variational learning algorithm and effective implementation of the optimism/pessimism principle. In particular, we propose a computationally efficient planning algorithm with UCB exploration by incorporating kernel embeddings of latent variable models.
  arXiv  Detail & Related papers  (2022-12-17T00:26:31Z)
• Multi-fidelity Gaussian Process for Biomanufacturing Process Modeling with Small Data [1.4687789417816917]
  We propose to use a statistical machine learning approach, multi-fidelity Gaussian process, for process modelling in biomanufacturing. We apply the multi-fidelity Gaussian process to solve two significant problems in biomanufacturing, bioreactor scale-up and knowledge transfer across cell lines, and demonstrate its efficacy on real-world datasets.
  arXiv  Detail & Related papers  (2022-11-26T06:38:34Z)
• Differentiable Agent-based Epidemiology [71.81552021144589]
  We introduce GradABM: a scalable, differentiable design for agent-based modeling that is amenable to gradient-based learning with automatic differentiation. GradABM can quickly simulate million-size populations in few seconds on commodity hardware, integrate with deep neural networks and ingest heterogeneous data sources.
  arXiv  Detail & Related papers  (2022-07-20T07:32:02Z)
• Extending Process Discovery with Model Complexity Optimization and Cyclic States Identification: Application to Healthcare Processes [62.997667081978825]
  The paper presents an approach to process mining providing semi-automatic support to model optimization. A model simplification approach is proposed, which essentially abstracts the raw model at the desired granularity. We aim to demonstrate the capabilities of the technological solution using three datasets from different applications in the healthcare domain.
  arXiv  Detail & Related papers  (2022-06-10T16:20:59Z)
• Multi-fidelity Hierarchical Neural Processes [79.0284780825048]
  Multi-fidelity surrogate modeling reduces the computational cost by fusing different simulation outputs. We propose Multi-fidelity Hierarchical Neural Processes (MF-HNP), a unified neural latent variable model for multi-fidelity surrogate modeling. We evaluate MF-HNP on epidemiology and climate modeling tasks, achieving competitive performance in terms of accuracy and uncertainty estimation.
  arXiv  Detail & Related papers  (2022-06-10T04:54:13Z)
• Opportunities of Hybrid Model-based Reinforcement Learning for Cell Therapy Manufacturing Process Development and Control [6.580930850408461]
  Key challenges of cell therapy manufacturing include high complexity, high uncertainty, and very limited process data. We propose a framework named "hybridRL" to efficiently guide process development and control. In the empirical study, cell therapy manufacturing examples are used to demonstrate that the proposed hybrid-RL framework can outperform the classical deterministic mechanistic model assisted process optimization.
  arXiv  Detail & Related papers  (2022-01-10T00:01:19Z)
• Towards an Automatic Analysis of CHO-K1 Suspension Growth in Microfluidic Single-cell Cultivation [63.94623495501023]
  We propose a novel Machine Learning architecture, which allows us to infuse a neural deep network with human-powered abstraction on the level of data. Specifically, we train a generative model simultaneously on natural and synthetic data, so that it learns a shared representation, from which a target variable, such as the cell count, can be reliably estimated.
  arXiv  Detail & Related papers  (2020-10-20T08:36:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.