Ro-SLM: Onboard Small Language Models for Robot Task Planning and Operation Code Generation

• URL: http://arxiv.org/abs/2604.10929v2
• Date: Tue, 14 Apr 2026 17:48:14 GMT
• Title: Ro-SLM: Onboard Small Language Models for Robot Task Planning and Operation Code Generation
• Abstract summary: Ro-SLM is a framework that enables reliable SLM-driven robot operation by distilling LLMs' knowledge and reasoning.<n>This paper introduces Ro-SLM, a framework that enables reliable SLM-driven robot operation by distilling LLMs' knowledge and reasoning.
• Score: 15.351149188703522
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Recent advances in large language models (LLMs) provide robots with contextual reasoning abilities to comprehend human instructions. Yet, current LLM-enabled robots typically depend on cloud-based models or high-performance computing infrastructure, which limit their deployment on robots under unreliable internet environments or with constrained computational resources, such as UAVs and small ground vehicles. Thus, deploying fine-tuned small language models (SLMs) that support onboard deployment offers a promising alternative. This paper introduces Ro-SLM, a framework that enables reliable SLM-driven robot operation by distilling LLMs' knowledge and reasoning. Ro-SLM starts from dataset synthesis by leveraging LLMs to generate diverse task instructions, produce corresponding ground truth code with minimal human assistance, and augment instructions into real-world application scenarios. Ro-SLM is then fine-tuned with the dataset, in which LLM serves as a reward function to guide the training. Extensive experiments on UAV operation tasks demonstrate that Ro-SLM improves the performance of SLM from being incapable of supporting robotic task planning and code generation to achieving performance that approaches LLM.

Related papers

• Hybrid LLM-based Intelligent Framework for Robot Task Scheduling [6.227803171257308]
  This study introduces intelligent frameworks that use Large Language Models (LLMs) to improve task scheduling for construction robots.<n>A well-balanced allocation strategy is developed, optimizing both time efficiency and resource utilization.
  arXiv  Detail & Related papers  (2026-05-15T00:04:38Z)
• An LLM-powered Natural-to-Robotic Language Translation Framework with Correctness Guarantees [40.31829665216206]
  Large Language Models (LLM) are increasingly being deployed in robotics to generate robot control programs for specific user tasks.<n>This paper introduces a natural-robotic language translation framework that provides correctness verification for generated control programs.
  arXiv  Detail & Related papers  (2025-08-26T14:32:49Z)
• Distilling On-device Language Models for Robot Planning with Minimal Human Intervention [116.93160528413655]
  PRISM is a framework for distilling small language model (SLM)-enabled robot planners.<n>We apply PRISM to three LLM-enabled planners for mapping and exploration, manipulation, and household assistance.<n>We demonstrate that PRISM improves the performance of Llama-3.2-3B from 10-20% of GPT-4o's performance to over 93%.
  arXiv  Detail & Related papers  (2025-06-20T21:44:27Z)
• MALMM: Multi-Agent Large Language Models for Zero-Shot Robotics Manipulation [62.854649499866774]
  Large Language Models (LLMs) have demonstrated remarkable planning abilities across various domains, including robotics manipulation and navigation.<n>We propose a novel multi-agent LLM framework that distributes high-level planning and low-level control code generation across specialized LLM agents.<n>We evaluate our approach on nine RLBench tasks, including long-horizon tasks, and demonstrate its ability to solve robotics manipulation in a zero-shot setting.
  arXiv  Detail & Related papers  (2024-11-26T17:53:44Z)
• DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Execution [114.61347672265076]
  Development of MLLMs for real-world robots is challenging due to the typically limited computation and memory capacities available on robotic platforms. We propose a Dynamic Early-Exit Framework for Robotic Vision-Language-Action Model (DeeR) that automatically adjusts the size of the activated MLLM. DeeR demonstrates significant reductions in computational costs of LLM by 5.2-6.5x and GPU memory of LLM by 2-6x without compromising performance.
  arXiv  Detail & Related papers  (2024-11-04T18:26:08Z)
• LLaRA: Supercharging Robot Learning Data for Vision-Language Policy [56.505551117094534]
  We introduce LLaRA: Large Language and Robotics Assistant, a framework that formulates robot action policy as visuo-textual conversations.<n>First, we present an automated pipeline to generate conversation-style instruction tuning data for robots from existing behavior cloning datasets.<n>We show that a VLM finetuned with a limited amount of such datasets can produce meaningful action decisions for robotic control.
  arXiv  Detail & Related papers  (2024-06-28T17:59:12Z)
• Robo-Instruct: Simulator-Augmented Instruction Alignment For Finetuning Code LLMs [34.16234805577773]
  We introduce ROBO-INSTRUCT, which synthesizes task-specific simulation environments on the fly during program execution.<n>ROBO-INSTRUCT integrates an LLM-aided post-processing procedure to refine instructions for better alignment with robot programs.
  arXiv  Detail & Related papers  (2024-05-30T15:47:54Z)
• Empowering Large Language Models on Robotic Manipulation with Affordance Prompting [23.318449345424725]
  Large language models fail to interact with the physical world by generating control sequences properly. Existing LLM-based approaches circumvent this problem by relying on additional pre-defined skills or pre-trained sub-policies. We propose a framework called LLM+A(ffordance) where the LLM serves as both the sub-task planner and the motion controller.
  arXiv  Detail & Related papers  (2024-04-17T03:06:32Z)
• Interactive Planning Using Large Language Models for Partially Observable Robotics Tasks [54.60571399091711]
  Large Language Models (LLMs) have achieved impressive results in creating robotic agents for performing open vocabulary tasks. We present an interactive planning technique for partially observable tasks using LLMs.
  arXiv  Detail & Related papers  (2023-12-11T22:54:44Z)
• Language to Rewards for Robotic Skill Synthesis [37.21434094015743]
  We introduce a new paradigm that harnesses large language models (LLMs) to define reward parameters that can be optimized and accomplish variety of robotic tasks. Using reward as the intermediate interface generated by LLMs, we can effectively bridge the gap between high-level language instructions or corrections to low-level robot actions.
  arXiv  Detail & Related papers  (2023-06-14T17:27:10Z)
• ProgPrompt: Generating Situated Robot Task Plans using Large Language Models [68.57918965060787]
  Large language models (LLMs) can be used to score potential next actions during task planning. We present a programmatic LLM prompt structure that enables plan generation functional across situated environments.
  arXiv  Detail & Related papers  (2022-09-22T20:29:49Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.