Autonomous Reasoning for Spacecraft Control: A Large Language Model Framework with Group Relative Policy Optimization

• URL: http://arxiv.org/abs/2601.04334v1
• Date: Wed, 07 Jan 2026 19:13:22 GMT
• Title: Autonomous Reasoning for Spacecraft Control: A Large Language Model Framework with Group Relative Policy Optimization
• Abstract summary: This paper presents a learning-based guidance-and-control approach that couples a reasoning-enabled Large Language Model (LLM) with Group Relative Policy Optimization ( GRPO)<n>A two-stage procedure consisting of Supervised Fine-Tuning (SFT) to learn formatting and control primitives, followed by GRPO for interaction-driven policy improvement, trains controllers for each environment.<n>Results demonstrate that an LLM with explicit reasoning, optimized via GRPO, can synthesize feasible stabilizing policies under consistent training settings across both linear and nonlinear systems.
• Score: 2.258690092379457
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: This paper presents a learning-based guidance-and-control approach that couples a reasoning-enabled Large Language Model (LLM) with Group Relative Policy Optimization (GRPO). A two-stage procedure consisting of Supervised Fine-Tuning (SFT) to learn formatting and control primitives, followed by GRPO for interaction-driven policy improvement, trains controllers for each environment. The framework is demonstrated on four control problems spanning a gradient of dynamical complexity, from canonical linear systems through nonlinear oscillatory dynamics to three-dimensional spacecraft attitude control with gyroscopic coupling and thrust constraints. Results demonstrate that an LLM with explicit reasoning, optimized via GRPO, can synthesize feasible stabilizing policies under consistent training settings across both linear and nonlinear systems. The two-stage training methodology enables models to generate control sequences while providing human-readable explanations of their decision-making process. This work establishes a foundation for applying GRPO-based reasoning to autonomous control systems, with potential applications in aerospace and other safety-critical domains.

Related papers

• Efficient Reinforcement Learning using Linear Koopman Dynamics for Nonlinear Robotic Systems [4.946483489399818]
  This paper presents a model-based reinforcement learning framework for optimal closed-loop control of nonlinear robotic systems.<n>The proposed approach learns linear lifted dynamics through Koopman operator theory and integrates the resulting model into an actor-critic architecture for policy optimization.<n>The framework is evaluated on several simulated nonlinear control benchmarks and two real-world hardware platforms.
  arXiv  Detail & Related papers  (2026-04-21T20:40:22Z)
• Subspace Control: Turning Constrained Model Steering into Controllable Spectral Optimization [27.280175507074322]
  We propose a subspace control framework for constrained model training.<n>SIFT (spectral interference-free training) consistently achieves substantial and robust performance improvements across all tasks.
  arXiv  Detail & Related papers  (2026-04-05T19:27:24Z)
• Plasma Shape Control via Zero-shot Generative Reinforcement Learning [17.3934551430283]
  We develop a novel framework for developing a versatile, zero-shot control policy from a large-scale offline dataset of PID-controlled discharges.<n>The resulting foundation policy can be deployed for diverse trajectory tracking tasks in a zero-shot manner without any task-specific fine-tuning.
  arXiv  Detail & Related papers  (2025-10-20T13:34:51Z)
• Receding Hamiltonian-Informed Optimal Neural Control and State Estimation for Closed-Loop Dynamical Systems [4.05766189327054]
  Hamiltonian-Informed Optimal Neural (Hion) controllers are a novel class of neural network-based controllers for dynamical systems.<n>Hion controllers estimate future states and develop an optimal control strategy using Pontryagin's Maximum Principle.
  arXiv  Detail & Related papers  (2024-11-02T16:06:29Z)
• Towards a Theoretical Foundation of Policy Optimization for Learning Control Policies [26.04704565406123]
  Gradient-based methods have been widely used for system design and optimization in diverse application domains. Recently, there has been a renewed interest in studying theoretical properties of these methods in the context of control and reinforcement learning. This article surveys some of the recent developments on policy optimization, a gradient-based iterative approach for feedback control synthesis.
  arXiv  Detail & Related papers  (2022-10-10T16:13:34Z)
• Comparative analysis of machine learning methods for active flow control [60.53767050487434]
  Genetic Programming (GP) and Reinforcement Learning (RL) are gaining popularity in flow control. This work presents a comparative analysis of the two, bench-marking some of their most representative algorithms against global optimization techniques.
  arXiv  Detail & Related papers  (2022-02-23T18:11:19Z)
• Policy Search for Model Predictive Control with Application to Agile Drone Flight [56.24908013905407]
  We propose a policy-search-for-model-predictive-control framework for MPC. Specifically, we formulate the MPC as a parameterized controller, where the hard-to-optimize decision variables are represented as high-level policies. Experiments show that our controller achieves robust and real-time control performance in both simulation and the real world.
  arXiv  Detail & Related papers  (2021-12-07T17:39:24Z)
• Sparsity in Partially Controllable Linear Systems [56.142264865866636]
  We study partially controllable linear dynamical systems specified by an underlying sparsity pattern. Our results characterize those state variables which are irrelevant for optimal control.
  arXiv  Detail & Related papers  (2021-10-12T16:41:47Z)
• Online Algorithms and Policies Using Adaptive and Machine Learning Approaches [0.22020053359163297]
  Two classes of nonlinear dynamic systems are considered, both of which are control-affine. We propose a combination of a Reinforcement Learning based policy in the outer loop suitably chosen to ensure stability and optimality for the nominal dynamics. In addition to establishing a stability guarantee with real-time control, the AC-RL controller is also shown to lead to parameter learning with persistent excitation.
  arXiv  Detail & Related papers  (2021-05-13T22:51:25Z)
• Gaussian Process-based Min-norm Stabilizing Controller for Control-Affine Systems with Uncertain Input Effects and Dynamics [90.81186513537777]
  We propose a novel compound kernel that captures the control-affine nature of the problem. We show that this resulting optimization problem is convex, and we call it Gaussian Process-based Control Lyapunov Function Second-Order Cone Program (GP-CLF-SOCP)
  arXiv  Detail & Related papers  (2020-11-14T01:27:32Z)
• Extended Radial Basis Function Controller for Reinforcement Learning [3.42658286826597]
  This paper proposes a hybrid reinforcement learning controller which dynamically interpolates a model-based linear controller and an arbitrary differentiable policy. The linear controller is designed based on local linearised model knowledge, and stabilises the system in a neighbourhood about an operating point. Learning has been done on both model-based (PILCO) and model-free (DDPG) frameworks.
  arXiv  Detail & Related papers  (2020-09-12T20:56:48Z)
• Adaptive Control and Regret Minimization in Linear Quadratic Gaussian (LQG) Setting [91.43582419264763]
  We propose LqgOpt, a novel reinforcement learning algorithm based on the principle of optimism in the face of uncertainty. LqgOpt efficiently explores the system dynamics, estimates the model parameters up to their confidence interval, and deploys the controller of the most optimistic model.
  arXiv  Detail & Related papers  (2020-03-12T19:56:38Z)
• Pontryagin Differentiable Programming: An End-to-End Learning and Control Framework [108.4560749465701]
  The Pontryagin Differentiable Programming methodology establishes a unified framework to solve a broad class of learning and control tasks. We investigate three learning modes of the PDP: inverse reinforcement learning, system identification, and control/planning. We demonstrate the capability of the PDP in each learning mode on different high-dimensional systems, including multi-link robot arm, 6-DoF maneuvering quadrotor, and 6-DoF rocket powered landing.
  arXiv  Detail & Related papers  (2019-12-30T15:35:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.