From LLM-Generated Specifications to Learned Quadruped Locomotion

• URL: http://arxiv.org/abs/2609.07111v1
• Date: Mon, 07 Sep 2026 06:53:53 GMT
• Title: From LLM-Generated Specifications to Learned Quadruped Locomotion
• Abstract summary: Quadruped robot locomotion policies are often trained using reinforcement learning.<n>It is often unclear which local rewards will induce the desired global behavior.<n>We study whether large language models (LLMs) can fill this gap by generating Signal Temporal Logic (PSTL) specifications.
• Score: 7.240440138551274
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Quadruped robot locomotion policies are often trained using reinforcement learning, which in turn relies heavily on hand-crafted reward functions. Designing reward functions requires substantial manual engineering, and it is often unclear which local rewards will induce the desired global behavior. Shaped rewards from formal specifications in languages like Signal Temporal Logic (STL) can make rewards more interpretable, but writing STL specifications itself still requires domain expertise. We study whether large language models (LLMs) can fill this gap by generating Parametric Signal Temporal Logic (PSTL) specifications that are subsequently used for policy learning. Given a natural language locomotion objective and a constrained specification grammar, GPT-5.5 and Qwen 3.6 independently propose STL templates for command tracking, safety, and gait structure. We instantiate the parameters of the generated PSTL templates using expert trajectories and retain only specifications that are consistent with demonstrated expert behavior. The resulting specifications are then transformed into smooth, finite-history reward functions and used to train a quadruped locomotion policy with Proximal Policy Optimization (PPO) in MuJoCo XLA (MJX). We evaluate both \emph{gait-aware} and \emph{gait-agnostic} settings. The former specifies walking-trot, trot, and bound regimes, while the latter allows contact patterns to emerge from the task objective. We compare against hand-engineered rewards, Text2Reward-style LLM-generated reward code, and an expert-switching oracle. Gait-aware Qwen 3.6 specifications achieved 100\% survival and command success across all tested speeds (0.3--2.1 m/s) and matched the target gait at high speeds, whereas Text2Reward achieved 0\% for both metrics at $\geq 1.9$ m/s. Videos: https://stl-locomotion.github.io/

Related papers

• Learning Gait-Aware Quadruped Locomotion with Temporal Logic Specifications [0.0]
  We introduce a framework where distinct gaits are specified using parameterized constraints expressed in Signal Temporal Logic.<n>We develop a reward shaping mechanism that provides learning agents a dense, continuous reward landscape that encodes desired behavior.<n>We instantiate the approach on Google's Barkour quadruped robot in MuJoCo XLA (MJX)
  arXiv  Detail & Related papers  (2026-07-01T04:57:39Z)
• LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling [63.679448814185456]
  Test-time scaling (TTS) has become an effective approach for improving large language model performance.<n>Existing TTS strategies are largely hand-crafted: researchers design patterns and tune them by intuition, leaving much of the computation-allocation space unexplored.<n>We propose an environment-driven framework, AutoTTS, that changes what researchers design: from individual TTSs to environments where TTS strategies can be discovered automatically.
  arXiv  Detail & Related papers  (2026-05-08T17:59:40Z)
• Detect Anything via Next Point Prediction [51.55967987350882]
  Rex- Omni is a 3B-scale MLLM that achieves state-of-the-art object perception performance.<n>On benchmarks like COCO and LVIS, Rex- Omni attains performance comparable to or exceeding regression-based models.
  arXiv  Detail & Related papers  (2025-10-14T17:59:54Z)
• Prompt-Based One-Shot Exact Length-Controlled Generation with LLMs [56.47577824219207]
  We present a prompt-based strategy that compels an off-the-shelf large language model to generate exactly a desired number of tokens.<n>The prompt appends countdown markers and explicit counting rules so that the model "writes while counting"<n>On MT-Bench-LI, strict length compliance with GPT-4.1 leaps from below 30% under naive prompts to above 95% with our countdown prompt.
  arXiv  Detail & Related papers  (2025-08-19T13:12:01Z)
• TeLoGraF: Temporal Logic Planning via Graph-encoded Flow Matching [13.154661571539577]
  We propose TeLoGraF, Temporal Logic Graph-encoded Flow, to learn solutions for general STL specifications.<n>We conduct experiments in five simulation environments ranging from simple dynamical models in the 2D space to high-dimensional 7DoF Franka Panda robot arm.<n>Our approach is 10-100X faster in inference and can work on any system dynamics.
  arXiv  Detail & Related papers  (2025-05-01T14:40:07Z)
• LTLDoG: Satisfying Temporally-Extended Symbolic Constraints for Safe Diffusion-based Planning [12.839846486863308]
  In this work, we focus on generating long-horizon trajectories that adhere to novel static and temporally-extended constraints/instructions at test time. We propose a data-driven diffusion-based framework, finiteDoG, that modifies the inference steps of the reverse process given an instruction specified using linear temporal logic. Experiments in robot navigation and manipulation illustrate that the method is able to generate trajectories that satisfy formulae that specify obstacle avoidance and visitation sequences.
  arXiv  Detail & Related papers  (2024-05-07T11:54:22Z)
• LeTI: Learning to Generate from Textual Interactions [60.425769582343506]
  We explore LMs' potential to learn from textual interactions (LETI) that not only check their correctness with binary labels but also pinpoint and explain errors in their outputs through textual feedback. Our focus is the code generation task, where the model produces code based on natural language instructions. LETI iteratively fine-tunes the model, using the objective LM, on a concatenation of natural language instructions, LM-generated programs, and textual feedback.
  arXiv  Detail & Related papers  (2023-05-17T15:53:31Z)
• Guiding Large Language Models via Directional Stimulus Prompting [114.84930073977672]
  We introduce Directional Stimulus Prompting, a novel framework for guiding black-box large language models (LLMs) toward specific desired outputs. Instead of directly adjusting LLMs, our method employs a small tunable policy model to generate an auxiliary directional stimulus prompt for each input instance.
  arXiv  Detail & Related papers  (2023-02-22T17:44:15Z)
• Self-Prompting Large Language Models for Zero-Shot Open-Domain QA [67.08732962244301]
  Open-Domain Question Answering (ODQA) aims to answer questions without explicitly providing background documents. This task becomes notably challenging in a zero-shot setting where no data is available to train tailored retrieval-reader models. We propose a Self-Prompting framework to explicitly utilize the massive knowledge encoded in the parameters of Large Language Models.
  arXiv  Detail & Related papers  (2022-12-16T18:23:43Z)
• Interactive Learning from Natural Language and Demonstrations using Signal Temporal Logic [5.88797764615148]
  Natural language (NL) is ambiguous, real world tasks and their safety requirements need to be communicated unambiguously. Signal Temporal Logic (STL) is a formal logic that can serve as a versatile, expressive, and unambiguous formal language to describe robotic tasks. We propose DIALOGUESTL, an interactive approach for learning correct and concise STL formulas from (often) ambiguous NL descriptions.
  arXiv  Detail & Related papers  (2022-07-01T19:08:43Z)
• Continuous Motion Planning with Temporal Logic Specifications using Deep Neural Networks [16.296473750342464]
  We propose a model-free reinforcement learning method to synthesize control policies for motion planning problems. The robot is modelled as a discrete Markovtime decision process (MDP) with continuous state and action spaces. We train deep neural networks to approximate the value function and policy using an actorcritic reinforcement learning method.
  arXiv  Detail & Related papers  (2020-04-02T17:58:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.