Towards Bridging the Gap between High-Level Reasoning and Execution on Robots

• URL: http://arxiv.org/abs/2401.00880v1
• Date: Sat, 30 Dec 2023 12:26:12 GMT
• Title: Towards Bridging the Gap between High-Level Reasoning and Execution on Robots
• Authors: Till Hofmann
• Abstract summary: When reasoning about actions, e.g., by means of task planning or agent programming with Golog, the robot's actions are typically modeled on an abstract level. However, when executing such an action on a robot it can no longer be seen as a primitive. In this thesis, we propose several approaches towards closing this gap.
• Score: 2.6107298043931206
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: When reasoning about actions, e.g., by means of task planning or agent programming with Golog, the robot's actions are typically modeled on an abstract level, where complex actions such as picking up an object are treated as atomic primitives with deterministic effects and preconditions that only depend on the current state. However, when executing such an action on a robot it can no longer be seen as a primitive. Instead, action execution is a complex task involving multiple steps with additional temporal preconditions and timing constraints. Furthermore, the action may be noisy, e.g., producing erroneous sensing results and not always having the desired effects. While these aspects are typically ignored in reasoning tasks, they need to be dealt with during execution. In this thesis, we propose several approaches towards closing this gap.

Related papers

• LTLf Synthesis on First-Order Action Theories [2.209921757303168]
  Golog is an expressive high-level agent language that includes nondeterministic operators. In this paper, we consider the more realistic case where parts of the non-determinism are under the control of the environment. A successful realization executes the program and satisfies the temporal goal for all possible environment actions.
  arXiv  Detail & Related papers  (2024-10-01T14:15:14Z)
• COHERENT: Collaboration of Heterogeneous Multi-Robot System with Large Language Models [49.24666980374751]
  COHERENT is a novel LLM-based task planning framework for collaboration of heterogeneous multi-robot systems. A Proposal-Execution-Feedback-Adjustment mechanism is designed to decompose and assign actions for individual robots. The experimental results show that our work surpasses the previous methods by a large margin in terms of success rate and execution efficiency.
  arXiv  Detail & Related papers  (2024-09-23T15:53:41Z)
• Task and Motion Planning for Execution in the Real [24.01204729304763]
  This work generates task and motion plans that include actions cannot be fully grounded at planning time. Execution combines offline planned motions and online behaviors till reaching the task goal. Forty real-robot trials and motivating demonstrations are performed to evaluate the proposed framework. Results show faster execution time, less number of actions, and more success in problems where diverse gaps arise.
  arXiv  Detail & Related papers  (2024-06-05T22:30:40Z)
• ThinkBot: Embodied Instruction Following with Thought Chain Reasoning [66.09880459084901]
  Embodied Instruction Following (EIF) requires agents to complete human instruction by interacting objects in complicated surrounding environments. We propose ThinkBot that reasons the thought chain in human instruction to recover the missing action descriptions. Our ThinkBot outperforms the state-of-the-art EIF methods by a sizable margin in both success rate and execution efficiency.
  arXiv  Detail & Related papers  (2023-12-12T08:30:09Z)
• Optimal task and motion planning and execution for human-robot multi-agent systems in dynamic environments [54.39292848359306]
  We propose a combined task and motion planning approach to optimize sequencing, assignment, and execution of tasks. The framework relies on decoupling tasks and actions, where an action is one possible geometric realization of a symbolic task. We demonstrate the approach effectiveness in a collaborative manufacturing scenario, in which a robotic arm and a human worker shall assemble a mosaic.
  arXiv  Detail & Related papers  (2023-03-27T01:50:45Z)
• TEACH: Temporal Action Composition for 3D Humans [50.97135662063117]
  Given a series of natural language descriptions, our task is to generate 3D human motions that correspond semantically to the text. In particular, our goal is to enable the synthesis of a series of actions, which we refer to as temporal action composition.
  arXiv  Detail & Related papers  (2022-09-09T00:33:40Z)
• Using Abstraction for Interpretable Robot Programs in Stochastic Domains [17.04153879817609]
  A robot's actions are inherently noisy, as its sensors are noisy and its actions do not always have the intended effects. Golog has been extended to models with degrees of belief and actions. The resulting programs are much harder to comprehend, because they need to deal with the noise. We define a high-level and nonstochastic model of the robot and then map the high-level model into the lower-level model.
  arXiv  Detail & Related papers  (2022-07-26T09:15:37Z)
• Controlling Golog Programs against MTL Constraints [4.56877715768796]
  We present an extension to Golog by clocks together with the required theoretical foundations as well as decidability results. We describe a method to synthesize a controller that executes both the high-level program and the low-level platform operations concurrently.
  arXiv  Detail & Related papers  (2022-04-07T17:16:37Z)
• A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Execution [54.385344986265714]
  We propose a persistent spatial semantic representation method to bridge the gap between language and robot actions. We evaluate our approach on the ALFRED benchmark and achieve state-of-the-art results, despite completely avoiding the commonly used step-by-step instructions.
  arXiv  Detail & Related papers  (2021-07-12T17:47:19Z)
• Thinking While Moving: Deep Reinforcement Learning with Concurrent Control [122.49572467292293]
  We study reinforcement learning in settings where sampling an action from the policy must be done concurrently with the time evolution of the controlled system. Much like a person or an animal, the robot must think and move at the same time, deciding on its next action before the previous one has completed.
  arXiv  Detail & Related papers  (2020-04-13T17:49:29Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.