Opportunities of Hybrid Model-based Reinforcement Learning for Cell Therapy Manufacturing Process Development and Control

• URL: http://arxiv.org/abs/2201.03116v1
• Date: Mon, 10 Jan 2022 00:01:19 GMT
• Title: Opportunities of Hybrid Model-based Reinforcement Learning for Cell Therapy Manufacturing Process Development and Control
• Authors: Hua Zheng, Wei Xie, Keqi Wang, Zheng Li
• Abstract summary: Key challenges of cell therapy manufacturing include high complexity, high uncertainty, and very limited process data. We propose a framework named "hybridRL" to efficiently guide process development and control. In the empirical study, cell therapy manufacturing examples are used to demonstrate that the proposed hybrid-RL framework can outperform the classical deterministic mechanistic model assisted process optimization.
• Score: 6.580930850408461
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Driven by the key challenges of cell therapy manufacturing, including high complexity, high uncertainty, and very limited process data, we propose a stochastic optimization framework named "hybrid-RL" to efficiently guide process development and control. We first create the bioprocess probabilistic knowledge graph that is a hybrid model characterizing the understanding of biomanufacturing process mechanisms and quantifying inherent stochasticity, such as batch-to-batch variation and bioprocess noise. It can capture the key features, including nonlinear reactions, time-varying kinetics, and partially observed bioprocess state. This hybrid model can leverage on existing mechanistic models and facilitate the learning from process data. Given limited process data, a computational sampling approach is used to generate posterior samples quantifying the model estimation uncertainty. Then, we introduce hybrid model-based Bayesian reinforcement learning (RL), accounting for both inherent stochasticity and model uncertainty, to guide optimal, robust, and interpretable decision making, which can overcome the key challenges of cell therapy manufacturing. In the empirical study, cell therapy manufacturing examples are used to demonstrate that the proposed hybrid-RL framework can outperform the classical deterministic mechanistic model assisted process optimization.

Related papers

• Digital Twin Calibration for Biological System-of-Systems: Cell Culture Manufacturing Process [3.0790370651488983]
  We consider the cell culture process multi-scale mechanistic model, also known as Biological System-of-Systems (Bio-SoS) This model with a modular design, composed of sub-models, allows us to integrate data across various production processes. To calibrate the Bio-SoS digital twin, we evaluate the mean squared error of model prediction and develop a computational approach to quantify the impact of parameter estimation error of individual sub-models on the prediction accuracy of digital twin.
  arXiv  Detail & Related papers  (2024-05-07T00:22:13Z)
• MFRL-BI: Design of a Model-free Reinforcement Learning Process Control Scheme by Using Bayesian Inference [5.375049126954924]
  Design of process control scheme is critical for quality assurance to reduce variations in manufacturing systems. We propose a model-free reinforcement learning (MFRL) approach to conduct experiments and optimize control simultaneously according to real-time data.
  arXiv  Detail & Related papers  (2023-09-17T08:18:55Z)
• Latent Variable Representation for Reinforcement Learning [131.03944557979725]
  It remains unclear theoretically and empirically how latent variable models may facilitate learning, planning, and exploration to improve the sample efficiency of model-based reinforcement learning. We provide a representation view of the latent variable models for state-action value functions, which allows both tractable variational learning algorithm and effective implementation of the optimism/pessimism principle. In particular, we propose a computationally efficient planning algorithm with UCB exploration by incorporating kernel embeddings of latent variable models.
  arXiv  Detail & Related papers  (2022-12-17T00:26:31Z)
• Multi-fidelity Gaussian Process for Biomanufacturing Process Modeling with Small Data [1.4687789417816917]
  We propose to use a statistical machine learning approach, multi-fidelity Gaussian process, for process modelling in biomanufacturing. We apply the multi-fidelity Gaussian process to solve two significant problems in biomanufacturing, bioreactor scale-up and knowledge transfer across cell lines, and demonstrate its efficacy on real-world datasets.
  arXiv  Detail & Related papers  (2022-11-26T06:38:34Z)
• Extending Process Discovery with Model Complexity Optimization and Cyclic States Identification: Application to Healthcare Processes [62.997667081978825]
  The paper presents an approach to process mining providing semi-automatic support to model optimization. A model simplification approach is proposed, which essentially abstracts the raw model at the desired granularity. We aim to demonstrate the capabilities of the technological solution using three datasets from different applications in the healthcare domain.
  arXiv  Detail & Related papers  (2022-06-10T16:20:59Z)
• Dynamic Bayesian Network Auxiliary ABC-SMC for Hybrid Model Bayesian Inference to Accelerate Biomanufacturing Process Mechanism Learning and Robust Control [2.727760379582405]
  We present a knowledge graph hybrid model characterizing complex causal interdependencies of underlying bioprocessing mechanisms. It can faithfully capture the important properties, including nonlinear reactions, partially observed state, and nonstationary dynamics. We derive a posterior distribution model uncertainty, which can facilitate mechanism learning and support robust process control.
  arXiv  Detail & Related papers  (2022-05-05T02:54:21Z)
• Predictive machine learning for prescriptive applications: a coupled training-validating approach [77.34726150561087]
  We propose a new method for training predictive machine learning models for prescriptive applications. This approach is based on tweaking the validation step in the standard training-validating-testing scheme. Several experiments with synthetic data demonstrate promising results in reducing the prescription costs in both deterministic and real models.
  arXiv  Detail & Related papers  (2021-10-22T15:03:20Z)
• Policy Optimization in Bayesian Network Hybrid Models of Biomanufacturing Processes [3.124775036986647]
  Biomanufacturing processes require close monitoring and control. We develop a novel model-based reinforcement learning framework that can achieve human-level control in low-data environments.
  arXiv  Detail & Related papers  (2021-05-13T20:39:02Z)
• Towards an Automatic Analysis of CHO-K1 Suspension Growth in Microfluidic Single-cell Cultivation [63.94623495501023]
  We propose a novel Machine Learning architecture, which allows us to infuse a neural deep network with human-powered abstraction on the level of data. Specifically, we train a generative model simultaneously on natural and synthetic data, so that it learns a shared representation, from which a target variable, such as the cell count, can be reliably estimated.
  arXiv  Detail & Related papers  (2020-10-20T08:36:51Z)
• Neuro-symbolic Neurodegenerative Disease Modeling as Probabilistic Programmed Deep Kernels [93.58854458951431]
  We present a probabilistic programmed deep kernel learning approach to personalized, predictive modeling of neurodegenerative diseases. Our analysis considers a spectrum of neural and symbolic machine learning approaches. We run evaluations on the problem of Alzheimer's disease prediction, yielding results that surpass deep learning.
  arXiv  Detail & Related papers  (2020-09-16T15:16:03Z)
• Predictive modeling approaches in laser-based material processing [59.04160452043105]
  This study aims to automate and forecast the effect of laser processing on material structures. The focus is centred on the performance of representative statistical and machine learning algorithms. Results can set the basis for a systematic methodology towards reducing material design, testing and production cost.
  arXiv  Detail & Related papers  (2020-06-13T17:28:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.