Robot Behavior-Tree-Based Task Generation with Large Language Models

• URL: http://arxiv.org/abs/2302.12927v1
• Date: Fri, 24 Feb 2023 22:53:10 GMT
• Title: Robot Behavior-Tree-Based Task Generation with Large Language Models
• Abstract summary: We propose a novel behavior-tree-based task generation approach that utilizes state-of-the-art large language models. We propose a Phase-Step prompt design that enables a hierarchical-structured robot task generation and further integrate it with behavior-tree-embedding-based search to set up the appropriate prompt. Our behavior-tree-based task generation approach does not require a set of pre-defined primitive tasks.
• Score: 14.384843227828775
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Nowadays, the behavior tree is gaining popularity as a representation for robot tasks due to its modularity and reusability. Designing behavior-tree tasks manually is time-consuming for robot end-users, thus there is a need for investigating automatic behavior-tree-based task generation. Prior behavior-tree-based task generation approaches focus on fixed primitive tasks and lack generalizability to new task domains. To cope with this issue, we propose a novel behavior-tree-based task generation approach that utilizes state-of-the-art large language models. We propose a Phase-Step prompt design that enables a hierarchical-structured robot task generation and further integrate it with behavior-tree-embedding-based search to set up the appropriate prompt. In this way, we enable an automatic and cross-domain behavior-tree task generation. Our behavior-tree-based task generation approach does not require a set of pre-defined primitive tasks. End-users only need to describe an abstract desired task and our proposed approach can swiftly generate the corresponding behavior tree. A full-process case study is provided to demonstrate our proposed approach. An ablation study is conducted to evaluate the effectiveness of our Phase-Step prompts. Assessment on Phase-Step prompts and the limitation of large language models are presented and discussed.

Related papers

• AlgoEvo: Self-Evolving Agentic Search for Automated Algorithm Discovery [25.96347096124607]
  AlgoEvo transforms automated algorithm discovery into an interactive, knowledge-accumulating process.<n>A design skill hub decouples paradigm-specific knowledge from the core discovery engine.<n>A hierarchical experience mechanism organizes search trajectories into a task-level tree to guide exploration.
  arXiv  Detail & Related papers  (2026-09-14T16:24:27Z)
• ScrambleToolBench: Agents Search Exhaustively Even When Their Own Map Points to the Next Step [74.83535434786145]
  We introduce ScrambleToolBench, an interactive terminal benchmark designed to isolate behavioral reasoning.<n>The benchmark requires agents to uncover hidden tool behaviors entirely through trial-and-error interaction.<n>Our evaluation of state-of-the-art language models reveals that successful initial discovery does not translate into robust adaptation.
  arXiv  Detail & Related papers  (2026-08-03T15:07:46Z)
• RoboWits: Unexpected Challenges for Robotic Creative Problem Solving [58.58727722988084]
  We introduce RoboWits, a bi-manual robotic benchmark designed to evaluate cognitive reasoning, creative tool use, and robustness to unexpected conditions.<n>Using the pipeline, we curated 30 diverse seed tasks and 208 tasks with mutations and graded difficulty across geometry, material, and assembly-based reasoning.<n>Our results reveal a significant performance gap: while pre-trained VLAs exhibit preliminary success on seed tasks after single-task fine-tuning, they struggle to perform on mutated tasks.
  arXiv  Detail & Related papers  (2026-05-28T17:57:15Z)
• DocOS: Towards Proactive Document-Guided Actions in GUI Agents [54.27655693145045]
  textbfDocOS is a benchmark designed to assess document-guided problem solving in fully interactive environments.<n>Experiments reveal that agents struggle to reliably locate relevant information during proactive search and frequently fail to faithfully ground retrieved instructions into precise actions.
  arXiv  Detail & Related papers  (2026-05-18T08:36:04Z)
• Adaptive Manipulation using Behavior Trees [12.061325774210392]
  We present the adaptive behavior tree, a scalable and generalizable behavior tree design. It enables a robot to quickly adapt to and learn from both visual and non-visual observations during task execution. We test our approach on a variety of tasks commonly found in industry.
  arXiv  Detail & Related papers  (2024-06-20T18:01:36Z)
• TA&AT: Enhancing Task-Oriented Dialog with Turn-Level Auxiliary Tasks and Action-Tree Based Scheduled Sampling [16.77137239284608]
  Task-oriented dialog systems have witnessed substantial progress due to conversational pre-training techniques. We propose turn-level multi-task objectives for the encoder. For the decoder, we introduce an action tree-based scheduled sampling technique.
  arXiv  Detail & Related papers  (2024-01-28T11:02:23Z)
• TapTree: Process-Tree Based Host Behavior Modeling and Threat Detection Framework via Sequential Pattern Mining [0.29465623430708915]
  This paper presents TapTree, an automated process-tree based technique to extract host behavior by compiling system events' semantic information. In our evaluation against a recent benchmark audit log dataset (DARPA OpTC), TapTree employs tree pattern queries and sequential pattern mining techniques to deduce the semantics of connected system events.
  arXiv  Detail & Related papers  (2023-12-10T15:12:55Z)
• Interactive Task Planning with Language Models [97.86399877812923]
  An interactive robot framework accomplishes long-horizon task planning and can easily generalize to new goals or distinct tasks, even during execution. Recent large language model based approaches can allow for more open-ended planning but often require heavy prompt engineering or domain-specific pretrained models. We propose a simple framework that achieves interactive task planning with language models.
  arXiv  Detail & Related papers  (2023-10-16T17:59:12Z)
• Tree-Planner: Efficient Close-loop Task Planning with Large Language Models [63.06270302774049]
  Tree-Planner reframes task planning with Large Language Models into three distinct phases. Tree-Planner achieves state-of-the-art performance while maintaining high efficiency.
  arXiv  Detail & Related papers  (2023-10-12T17:59:50Z)
• Unsupervised Task Graph Generation from Instructional Video Transcripts [53.54435048879365]
  We consider a setting where text transcripts of instructional videos performing a real-world activity are provided. The goal is to identify the key steps relevant to the task as well as the dependency relationship between these key steps. We propose a novel task graph generation approach that combines the reasoning capabilities of instruction-tuned language models along with clustering and ranking components.
  arXiv  Detail & Related papers  (2023-02-17T22:50:08Z)
• Build generally reusable agent-environment interaction models [28.577502598559988]
  This paper tackles the problem of how to pre-train a model and make it generally reusable backbones for downstream task learning. We propose a method that builds an agent-environment interaction model by learning domain invariant successor features from the agent's vast experiences covering various tasks, then discretize them into behavior prototypes. We provide preliminary results that show downstream task learning based on a pre-trained embodied set structure can handle unseen changes in task objectives, environmental dynamics and sensor modalities.
  arXiv  Detail & Related papers  (2022-11-13T07:33:14Z)
• Verifying Safety of Behaviour Trees in Event-B [0.0]
  Behavior Trees (BT) are becoming increasingly popular in the robotics community. We propose a formal specification of Behavior Trees and a methodology to prove invariants of already used trees.
  arXiv  Detail & Related papers  (2022-09-28T12:26:50Z)
• Fast Inference and Transfer of Compositional Task Structures for Few-shot Task Generalization [101.72755769194677]
  We formulate it as a few-shot reinforcement learning problem where a task is characterized by a subtask graph. Our multi-task subtask graph inferencer (MTSGI) first infers the common high-level task structure in terms of the subtask graph from the training tasks. Our experiment results on 2D grid-world and complex web navigation domains show that the proposed method can learn and leverage the common underlying structure of the tasks for faster adaptation to the unseen tasks.
  arXiv  Detail & Related papers  (2022-05-25T10:44:25Z)
• Can Foundation Models Perform Zero-Shot Task Specification For Robot Manipulation? [54.442692221567796]
  Task specification is critical for engagement of non-expert end-users and adoption of personalized robots. A widely studied approach to task specification is through goals, using either compact state vectors or goal images from the same robot scene. In this work, we explore alternate and more general forms of goal specification that are expected to be easier for humans to specify and use.
  arXiv  Detail & Related papers  (2022-04-23T19:39:49Z)
• A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Execution [54.385344986265714]
  We propose a persistent spatial semantic representation method to bridge the gap between language and robot actions. We evaluate our approach on the ALFRED benchmark and achieve state-of-the-art results, despite completely avoiding the commonly used step-by-step instructions.
  arXiv  Detail & Related papers  (2021-07-12T17:47:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.