Hybrid Reinforcement Learning and Model Predictive Control for Adaptive Control of Hydrogen-Diesel Dual-Fuel Combustion

• URL: http://arxiv.org/abs/2504.16875v1
• Date: Wed, 23 Apr 2025 16:51:49 GMT
• Title: Hybrid Reinforcement Learning and Model Predictive Control for Adaptive Control of Hydrogen-Diesel Dual-Fuel Combustion
• Authors: Julian Bedei, Murray McBain, Charles Robert Koch, Jakob Andert, David Gordon,
• Abstract summary: Reinforcement Learning (RL) and Machine Learning Integrated Model Predictive Control (ML-MPC) are promising approaches for optimizing hydrogen-diesel dual-fuel engine control.<n>This study proposes a hybrid RL and ML-MPC approach that uses an ML-MPC framework while incorporating an RL agent.
• Score: 1.188383832081829
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Reinforcement Learning (RL) and Machine Learning Integrated Model Predictive Control (ML-MPC) are promising approaches for optimizing hydrogen-diesel dual-fuel engine control, as they can effectively control multiple-input multiple-output systems and nonlinear processes. ML-MPC is advantageous for providing safe and optimal controls, ensuring the engine operates within predefined safety limits. In contrast, RL is distinguished by its adaptability to changing conditions through its learning-based approach. However, the practical implementation of either method alone poses challenges. RL requires high variance in control inputs during early learning phases, which can pose risks to the system by potentially executing unsafe actions, leading to mechanical damage. Conversely, ML-MPC relies on an accurate system model to generate optimal control inputs and has limited adaptability to system drifts, such as injector aging, which naturally occur in engine applications. To address these limitations, this study proposes a hybrid RL and ML-MPC approach that uses an ML-MPC framework while incorporating an RL agent to dynamically adjust the ML-MPC load tracking reference in response to changes in the environment. At the same time, the ML-MPC ensures that actions stay safe throughout the RL agent's exploration. To evaluate the effectiveness of this approach, fuel pressure is deliberately varied to introduce a model-plant mismatch between the ML-MPC and the engine test bench. The result of this mismatch is a root mean square error (RMSE) in indicated mean effective pressure of 0.57 bar when running the ML-MPC. The experimental results demonstrate that RL successfully adapts to changing boundary conditions by altering the tracking reference while ML-MPC ensures safe control inputs. The quantitative improvement in load tracking by implementing RL is an RSME of 0.44 bar.

Related papers

• Enhancing Autonomous Driving Systems with On-Board Deployed Large Language Models [25.418353477628035]
  This work proposes a hybrid architecture combining low-level Model Predictive Controller (MPC) with locally deployed Large Language Models (LLMs) DecisionxLLM module evaluates robotic state information against natural language instructions to ensure adherence to desired driving behavior. We propose an approach that exploits Retrieval Augmented Generation (RAG), Low Rank Adaptation (LoRA), fine-tuning, and quantization.
  arXiv  Detail & Related papers  (2025-04-15T13:49:17Z)
• Safe Reinforcement Learning for Real-World Engine Control [39.9074966439168]
  This work introduces a toolchain for applying Reinforcement Learning (RL) to safety-critical real-world environments.<n>RL provides a viable solution, however, safety concerns, such as excessive pressure rise rates, must be addressed.<n>Real-time safety monitoring based on the k-nearest neighbor algorithm is implemented, enabling safe interaction with the testbench.
  arXiv  Detail & Related papers  (2025-01-28T01:19:05Z)
• Self-Healing Machine Learning: A Framework for Autonomous Adaptation in Real-World Environments [50.310636905746975]
  Real-world machine learning systems often encounter model performance degradation due to distributional shifts in the underlying data generating process. Existing approaches to addressing shifts, such as concept drift adaptation, are limited by their reason-agnostic nature. We propose self-healing machine learning (SHML) to overcome these limitations.
  arXiv  Detail & Related papers  (2024-10-31T20:05:51Z)
• Safety Constrained Multi-Agent Reinforcement Learning for Active Voltage Control [34.95810473913879]
  We formalize the active voltage control problem as a constrained Markov game and propose a safety-constrained MARL algorithm. We evaluate our approach in the power distribution network simulation environment with real-world scale scenarios.
  arXiv  Detail & Related papers  (2024-05-14T09:03:00Z)
• Parameter-Adaptive Approximate MPC: Tuning Neural-Network Controllers without Retraining [50.00291020618743]
  This work introduces a novel, parameter-adaptive AMPC architecture capable of online tuning without recomputing large datasets and retraining. We showcase the effectiveness of parameter-adaptive AMPC by controlling the swing-ups of two different real cartpole systems with a severely resource-constrained microcontroller (MCU) Taken together, these contributions represent a marked step toward the practical application of AMPC in real-world systems.
  arXiv  Detail & Related papers  (2024-04-08T20:02:19Z)
• Reinforcement Learning with Model Predictive Control for Highway Ramp Metering [14.389086937116582]
  This work explores the synergy between model-based and learning-based strategies to enhance traffic flow management.<n>The control problem is formulated as an RL task by crafting a suitable stage cost function.<n>An MPC-based RL approach, which leverages the MPC optimal problem as a function approximation for the RL algorithm, is proposed to learn to efficiently control an on-ramp.
  arXiv  Detail & Related papers  (2023-11-15T09:50:54Z)
• Introducing a Deep Neural Network-based Model Predictive Control Framework for Rapid Controller Implementation [41.38091115195305]
  This work presents the experimental implementation of a deep neural network (DNN) based nonlinear MPC for Homogeneous Charge Compression Ignition (HCCI) combustion control. Using the acados software package to enable the real-time implementation of the MPC on an ARM Cortex A72, the optimization calculations are completed within 1.4 ms. The IMEP trajectory following of the developed controller was excellent, with a root-mean-square error of 0.133 bar, in addition to observing process constraints.
  arXiv  Detail & Related papers  (2023-10-12T15:03:50Z)
• Learned Risk Metric Maps for Kinodynamic Systems [54.49871675894546]
  We present Learned Risk Metric Maps for real-time estimation of coherent risk metrics of high dimensional dynamical systems. LRMM models are simple to design and train, requiring only procedural generation of obstacle sets, state and control sampling, and supervised training of a function approximator.
  arXiv  Detail & Related papers  (2023-02-28T17:51:43Z)
• Train Hard, Fight Easy: Robust Meta Reinforcement Learning [78.16589993684698]
  A major challenge of reinforcement learning (RL) in real-world applications is the variation between environments, tasks or clients. Standard MRL methods optimize the average return over tasks, but often suffer from poor results in tasks of high risk or difficulty. In this work, we define a robust MRL objective with a controlled level. The data inefficiency is addressed via the novel Robust Meta RL algorithm (RoML)
  arXiv  Detail & Related papers  (2023-01-26T14:54:39Z)
• Covert Model Poisoning Against Federated Learning: Algorithm Design and Optimization [76.51980153902774]
  Federated learning (FL) is vulnerable to external attacks on FL models during parameters transmissions. In this paper, we propose effective MP algorithms to combat state-of-the-art defensive aggregation mechanisms. Our experimental results demonstrate that the proposed CMP algorithms are effective and substantially outperform existing attack mechanisms.
  arXiv  Detail & Related papers  (2021-01-28T03:28:18Z)
• Guided Constrained Policy Optimization for Dynamic Quadrupedal Robot Locomotion [78.46388769788405]
  We introduce guided constrained policy optimization (GCPO), an RL framework based upon our implementation of constrained policy optimization (CPPO) We show that guided constrained RL offers faster convergence close to the desired optimum resulting in an optimal, yet physically feasible, robotic control behavior without the need for precise reward function tuning.
  arXiv  Detail & Related papers  (2020-02-22T10:15:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.