Knowledge-Distilled End-to-End Reinforcement Learning for Smooth 6-DOF Thrust Control and Rapid Adaptation to Ocean Currents in Remotely Operated Vehicles

• URL: http://arxiv.org/abs/2608.08598v1
• Date: Sun, 09 Aug 2026 09:23:15 GMT
• Title: Knowledge-Distilled End-to-End Reinforcement Learning for Smooth 6-DOF Thrust Control and Rapid Adaptation to Ocean Currents in Remotely Operated Vehicles
• Abstract summary: This paper proposes the thrust rapid current adaptation policy optimization (TSRCA-PPO) method which learns a near-optimal strategy by a twostage distillation learning framework.<n> Simulation results demonstrate that the proposed TSRCA-PPO method consistently outperforms the conventional cascaded P-PID controller.
• Score: 16.40999985739862
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: With the continuous improvement of computational capabilities, end-to-end reinforcement learning has been rapidly developed for remotely operated vehicles control. Nevertheless, existing end-to-end reinforcement-learningbased methods still face challenges in achieving optimal control under oceancurrent disturbances. In particular, there remains a lack of a unified control framework that can simultaneously achieve low steady-state tracking error, rapid transient response, energy-efficient operation, and smooth controlforce outputs under disturbances. To address the issue, this paper proposes the thrust smoothness rapid current adaptation proximal policy optimization (TSRCA-PPO) method which learns a near-optimal strategy by a twostage distillation learning framework. The core innovations of this work lie in the reward-function design and the privileged multi-encoder architecture. Ablation studies validate the effectiveness of each module. Simulation results demonstrate that the proposed TSRCA-PPO method consistently outperforms the conventional cascaded P-PID controller across all evaluation metrics. Specifically, TSRCA-PPO reduces the steady-state position error, steady-state attitude error, settling time, energy index, and thrustsmoothness index to 42.7%, 76.5%, 10.6%, 93.5%, and 15.9% of the corresponding P-PID values, respectively.

Related papers

• Reinforcement Learning with Inner-loop Dynamics Estimator for Aerial Manipulation under Uncertainty [0.560928143828791]
  We present a hierarchical control framework that combines Reinforcement Learning (RL) with an inner-loop dynamics estimator to address this problem.<n>We validate the proposed approach on a custom quadrotor equipped with a 3-DoF manipulator through hardware experiments under varying payload conditions.
  arXiv  Detail & Related papers  (2026-06-15T12:16:01Z)
• A Heuristic Approach for Performance Tuning in RL-based Quadrotor Control via Reward Design and Termination Conditions [14.576960188691215]
  Reinforcement learning (RL)-based quadrotor control policies have achieved impressive performance in tasks such as fast navigation in cluttered environments and drone racing.<n>We present a novel approach to achieve tunable performance in RL-based Quadrotor control through reward design and termination conditions.
  arXiv  Detail & Related papers  (2026-05-18T22:40:30Z)
• Drift is a Sampling Error: SNR-Aware Power Distributions for Long-Horizon Robotic Planning [6.615277432526784]
  We propose Context-Aware Power Sampling (CAPS), a training-free inference-time computation framework.<n>CAPS sharpens global trajectory probabilities, enabling lookahead search over the model's conditional generative trajectory distribution.<n> experiments on RoboTwin, Simpler-WindowX, and Libero-long benchmarks show that CAPS achieves substantial improvements over strong baselines.
  arXiv  Detail & Related papers  (2026-05-10T13:49:56Z)
• Unifying Group-Relative and Self-Distillation Policy Optimization via Sample Routing [79.88256756334327]
  Self-Distillation Policy Optimization (SDPO) addresses this by providing denser, more targeted logit-level supervision.<n>We propose Sample-Routed Policy Optimization (SRPO)<n>SRPO routes correct samples to GRPO's reward-aligned reinforcement and failed samples to SDPO's targeted logit-level correction.
  arXiv  Detail & Related papers  (2026-04-02T17:29:18Z)
• Robust Spatiotemporal Motion Planning for Multi-Agent Autonomous Racing via Topological Gap Identification and Accelerated MPC [30.927534234540502]
  Current methods often oversimplify or abandon strict kinematic constraints.<n>We propose a Topological Identification and Accelerated MPC framework.<n>It reduces total maneuver time by 51.6% in sequential scenarios.<n>It consistently maintains an overtaking success rate exceeding 81% in dense bottlenecks.
  arXiv  Detail & Related papers  (2026-03-10T04:55:30Z)
• When Sensors Fail: Temporal Sequence Models for Robust PPO under Sensor Drift [64.37959940809633]
  We study robustness of Proximal Policy Optimization (PPO) under temporally persistent sensor failures.<n>We show Transformer-based sequence policies substantially outperform, RNN, and SSMs in robustness, maintaining high returns even when large fractions of sensors are unavailable.
  arXiv  Detail & Related papers  (2026-03-04T22:21:54Z)
• Constrained PSO Six-Parameter Fuzzy PID Tuning Method for Balanced Optimization of Depth Tracking Performance in Underwater Vehicles [0.0]
  This paper proposes a constrained particle swarm optimization (PSO) method for tuning fuzzy PID controllers.<n>It achieves synergistic optimization of the overall tuning strength and dynamic response characteristics of the fuzzy PID system.
  arXiv  Detail & Related papers  (2026-02-13T08:11:18Z)
• HPTune: Hierarchical Proactive Tuning for Collision-Free Model Predictive Control [55.015741360766945]
  We propose a hierarchical proactive tuning (HPTune) framework that combines both a fast-level tuning and a slow-level tuning.<n>We integrate HPTune with the Doppler LiDAR that provides obstacle velocities apart from position-only measurements for enhanced motion predictions.<n>It is found that HPTune enables situation-tailored motion planning by formulating a safe, agile collision avoidance strategy.
  arXiv  Detail & Related papers  (2026-01-29T07:15:39Z)
• Iterative Refinement of Flow Policies in Probability Space for Online Reinforcement Learning [56.47948583452555]
  We introduce the Stepwise Flow Policy (SWFP) framework, founded on the key insight that discretizing the flow matching inference process via a fixed-step Euler scheme aligns it with the variational Jordan-Kinderlehrer-Otto principle from optimal transport.<n>SWFP decomposes the global flow into a sequence of small, incremental transformations between proximate distributions.<n>This decomposition yields an efficient algorithm that fine-tunes pre-trained flows via a cascade of small flow blocks, offering significant advantages.
  arXiv  Detail & Related papers  (2025-10-17T07:43:51Z)
• Growing Q-Networks: Solving Continuous Control Tasks with Adaptive Control Resolution [51.83951489847344]
  In robotics applications, smooth control signals are commonly preferred to reduce system wear and energy efficiency. In this work, we aim to bridge this performance gap by growing discrete action spaces from coarse to fine control resolution. Our work indicates that an adaptive control resolution in combination with value decomposition yields simple critic-only algorithms that yield surprisingly strong performance on continuous control tasks.
  arXiv  Detail & Related papers  (2024-04-05T17:58:37Z)
• Confidence-Controlled Exploration: Efficient Sparse-Reward Policy Learning for Robot Navigation [72.24964965882783]
  Reinforcement learning (RL) is a promising approach for robotic navigation, allowing robots to learn through trial and error.<n>Real-world robotic tasks often suffer from sparse rewards, leading to inefficient exploration and suboptimal policies.<n>We introduce Confidence-Controlled Exploration (CCE), a novel method that improves sample efficiency in RL-based robotic navigation without modifying the reward function.
  arXiv  Detail & Related papers  (2023-06-09T18:45:15Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.