A Heuristic Approach for Performance Tuning in RL-based Quadrotor Control via Reward Design and Termination Conditions

• URL: http://arxiv.org/abs/2605.19166v1
• Date: Mon, 18 May 2026 22:40:30 GMT
• Title: A Heuristic Approach for Performance Tuning in RL-based Quadrotor Control via Reward Design and Termination Conditions
• Abstract summary: Reinforcement learning (RL)-based quadrotor control policies have achieved impressive performance in tasks such as fast navigation in cluttered environments and drone racing.<n>We present a novel approach to achieve tunable performance in RL-based Quadrotor control through reward design and termination conditions.
• Score: 14.576960188691215
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Reinforcement learning (RL)-based quadrotor control policies have achieved impressive performance in tasks such as fast navigation in cluttered environments and drone racing, where the focus is on speed and agility. However, in several applications, such as infrastructure inspection, it is critical to achieve precise, controlled maneuvers with tunable performance. In this article, we present a novel heuristic approach to achieve tunable performance in RL-based Quadrotor control through reward design and termination conditions. We present a novel reward structure containing dual bandwidth exponentials that achieves a baseline critically damped response in setpoint tracking, with low steady-state errors. When trained with a Proximal Policy Optimization (PPO) algorithm, in conjunction with episode truncation conditions, the desired performance is achieved in 6 million time steps in a sample-efficient manner. In order to tune the performance about the baseline behavior, we present intuitive heuristic rules to adjust the reward weights and exponential coefficients to achieve faster (acrobatic-like) and slower (inspection-like) settling time performance, while retaining the baseline critically damped response and approximately 2\% steady-state error. We evaluate the three RL policies (baseline, acrobatic, and inspection) across 100 trials and show accurate and tunable performance in position and yaw tracking from random initial conditions, thereby demonstrating the effectiveness of the proposed heuristic approach.

Related papers

• Knowledge-Distilled End-to-End Reinforcement Learning for Smooth 6-DOF Thrust Control and Rapid Adaptation to Ocean Currents in Remotely Operated Vehicles [16.40999985739862]
  This paper proposes the thrust rapid current adaptation policy optimization (TSRCA-PPO) method which learns a near-optimal strategy by a twostage distillation learning framework.<n> Simulation results demonstrate that the proposed TSRCA-PPO method consistently outperforms the conventional cascaded P-PID controller.
  arXiv  Detail & Related papers  (2026-08-09T09:23:15Z)
• Test-Time Gradient Guidance of Flow Policies in Reinforcement Learning [50.738952715864116]
  Expressive continuous control policies form the backbone of advances in scaling imitation learning for simulated and real robot control.<n>We propose QGF (Q-Guided Flow), an RL algorithm that performs policy optimization entirely at test time.<n> Empirically, QGF outperforms prior test-time RL methods on single-task and goal-conditioned offline RL benchmarks.
  arXiv  Detail & Related papers  (2026-06-09T16:45:57Z)
• Benchmarking Empirical and Learning-Based Approaches for Feedforward Steering Control in Autonomous Racing [2.183305325115666]
  Feedforward steering control is a key component of hierarchical control architectures for autonomous racing.<n>This paper presents a benchmark of two learning-based and two empirical feedforward controllers.<n>We test the feedforward controllers in a high-fidelity simulation framework based on the real-world Abu Dhabi Autonomous Racing League competition.
  arXiv  Detail & Related papers  (2026-05-20T12:44:36Z)
• Spatially-Aware Adaptive Trajectory Optimization with Controller-Guided Feedback for Autonomous Racing [74.83272587893508]
  We present a framework for autonomous raceline optimization that combines NURBS-based trajectory representation, CMA-ES global trajectory optimization, and controller-guided spatial feedback.<n>In simulation, our approach achieves a 17.38% lap time reduction compared to a controller parametrized with maximum static acceleration.<n>On real hardware, tested with different tire compounds ranging from high to low friction, we obtain a 7.60% lap time improvement without explicitly parametrizing friction.
  arXiv  Detail & Related papers  (2026-02-17T15:10:44Z)
• Conformal Signal Temporal Logic for Robust Reinforcement Learning Control: A Case Study [0.0]
  We investigate how formal temporal logic specifications can enhance the safety and robustness of reinforcement learning control in aerospace applications.<n>Experiments show that the conformal shield preserves STL satisfaction while maintaining near baseline performance.
  arXiv  Detail & Related papers  (2026-02-15T22:10:11Z)
• CVeDRL: An Efficient Code Verifier via Difficulty-aware Reinforcement Learning [57.24524263804788]
  Code verifiers play a critical role in post-verification for LLM-based code generation.<n>Existing supervised fine-tuning methods suffer from data scarcity, high failure rates, and poor inference efficiency.<n>We show that naive RL with only functionality rewards fails to generate effective unit tests for difficult branches and samples.
  arXiv  Detail & Related papers  (2026-01-30T10:33:29Z)
• HPTune: Hierarchical Proactive Tuning for Collision-Free Model Predictive Control [55.015741360766945]
  We propose a hierarchical proactive tuning (HPTune) framework that combines both a fast-level tuning and a slow-level tuning.<n>We integrate HPTune with the Doppler LiDAR that provides obstacle velocities apart from position-only measurements for enhanced motion predictions.<n>It is found that HPTune enables situation-tailored motion planning by formulating a safe, agile collision avoidance strategy.
  arXiv  Detail & Related papers  (2026-01-29T07:15:39Z)
• Performance-driven Constrained Optimal Auto-Tuner for MPC [36.143463447995536]
  We propose COAT-MPC, Constrained Optimal Auto-Tuner for MPC.<n>COAT-MPC gathers performance data and learns by updating its posterior belief.<n>We theoretically analyze COAT-MPC, showing that it satisfies performance constraints with arbitrarily high probability.
  arXiv  Detail & Related papers  (2025-03-10T09:56:08Z)
• Safe Load Balancing in Software-Defined-Networking [1.2521494095948067]
  Control Barrier (CBF) designed on top of Deep Reinforcement Learning (DRL) algorithms for load-balancing. We show that our DRL-CBF approach is capable of meeting safety requirements during training and testing.
  arXiv  Detail & Related papers  (2024-10-22T09:34:22Z)
• Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolution [51.83951489847344]
  In robotics applications, smooth control signals are commonly preferred to reduce system wear and energy efficiency. In this work, we aim to bridge this performance gap by growing discrete action spaces from coarse to fine control resolution. Our work indicates that an adaptive control resolution in combination with value decomposition yields simple critic-only algorithms that yield surprisingly strong performance on continuous control tasks.
  arXiv  Detail & Related papers  (2024-04-05T17:58:37Z)
• Posterior Sampling with Delayed Feedback for Reinforcement Learning with Linear Function Approximation [62.969796245827006]
  Delayed-PSVI is an optimistic value-based algorithm that explores the value function space via noise perturbation with posterior sampling. We show our algorithm achieves $widetildeO(sqrtd3H3 T + d2H2 E[tau]$ worst-case regret in the presence of unknown delays. We incorporate a gradient-based approximate sampling scheme via Langevin dynamics for Delayed-LPSVI.
  arXiv  Detail & Related papers  (2023-10-29T06:12:43Z)
• Performance-Driven Controller Tuning via Derivative-Free Reinforcement Learning [6.5158195776494]
  We tackle the controller tuning problem using a novel derivative-free reinforcement learning framework. We conduct numerical experiments on two concrete examples from autonomous driving, namely, adaptive cruise control with PID controller and trajectory tracking with MPC controller. Experimental results show that the proposed method outperforms popular baselines and highlight its strong potential for controller tuning.
  arXiv  Detail & Related papers  (2022-09-11T13:01:14Z)
• Guided Constrained Policy Optimization for Dynamic Quadrupedal Robot Locomotion [78.46388769788405]
  We introduce guided constrained policy optimization (GCPO), an RL framework based upon our implementation of constrained policy optimization (CPPO) We show that guided constrained RL offers faster convergence close to the desired optimum resulting in an optimal, yet physically feasible, robotic control behavior without the need for precise reward function tuning.
  arXiv  Detail & Related papers  (2020-02-22T10:15:53Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.