Enabling human-like task identification from natural conversation
- URL: http://arxiv.org/abs/2008.10073v2
- Date: Sat, 29 Aug 2020 04:54:20 GMT
- Title: Enabling human-like task identification from natural conversation
- Authors: Pradip Pramanick, Chayan Sarkar, Balamuralidhar P, Ajay Kattepur,
Indrajit Bhattacharya, Arpan Pal
- Abstract summary: We provide a non-trivial method to combine an NLP engine and a planner such that a robot can successfully identify tasks and all the relevant parameters and generate an accurate plan for the task.
This work makes a significant stride towards enabling a human-like task understanding capability in a robot.
- Score: 7.00597813134145
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: A robot as a coworker or a cohabitant is becoming mainstream day-by-day with
the development of low-cost sophisticated hardware. However, an accompanying
software stack that can aid the usability of the robotic hardware remains the
bottleneck of the process, especially if the robot is not dedicated to a single
job. Programming a multi-purpose robot requires an on the fly mission
scheduling capability that involves task identification and plan generation.
The problem dimension increases if the robot accepts tasks from a human in
natural language. Though recent advances in NLP and planner development can
solve a variety of complex problems, their amalgamation for a dynamic robotic
task handler is used in a limited scope. Specifically, the problem of
formulating a planning problem from natural language instructions is not
studied in details. In this work, we provide a non-trivial method to combine an
NLP engine and a planner such that a robot can successfully identify tasks and
all the relevant parameters and generate an accurate plan for the task.
Additionally, some mechanism is required to resolve the ambiguity or missing
pieces of information in natural language instruction. Thus, we also develop a
dialogue strategy that aims to gather additional information with minimal
question-answer iterations and only when it is necessary. This work makes a
significant stride towards enabling a human-like task understanding capability
in a robot.
Related papers
- Safe Task Planning for Language-Instructed Multi-Robot Systems using Conformal Prediction [11.614036749291216]
We introduce a new decentralized multi-robot planner called S-ATLAS for Safe plAnning for Teams of Language-instructed AgentS.
We show that the proposed planner can achieve user-specified task success rates while minimizing the overall number of help requests.
arXiv Detail & Related papers (2024-02-23T15:02:44Z) - RoboScript: Code Generation for Free-Form Manipulation Tasks across Real
and Simulation [77.41969287400977]
This paper presents textbfRobotScript, a platform for a deployable robot manipulation pipeline powered by code generation.
We also present a benchmark for a code generation benchmark for robot manipulation tasks in free-form natural language.
We demonstrate the adaptability of our code generation framework across multiple robot embodiments, including the Franka and UR5 robot arms.
arXiv Detail & Related papers (2024-02-22T15:12:00Z) - Automated Process Planning Based on a Semantic Capability Model and SMT [50.76251195257306]
In research of manufacturing systems and autonomous robots, the term capability is used for a machine-interpretable specification of a system function.
We present an approach that combines these two topics: starting from a semantic capability model, an AI planning problem is automatically generated.
arXiv Detail & Related papers (2023-12-14T10:37:34Z) - Natural Language Instructions for Intuitive Human Interaction with
Robotic Assistants in Field Construction Work [4.223718588030052]
This paper proposes a framework to allow human workers to interact with construction robots based on natural language instructions.
The proposed method consists of three stages: Natural Language Understanding (NLU), Information Mapping (IM), and Robot Control (RC)
arXiv Detail & Related papers (2023-07-09T15:02:34Z) - Logic programming for deliberative robotic task planning [2.610470075814367]
We present a survey on recent advances in the application of logic programming to the problem of task planning.
We analyze different planners and their suitability for specific robotic applications, based on expressivity in domain representation, computational efficiency and software implementation.
arXiv Detail & Related papers (2023-01-18T14:11:55Z) - ProgPrompt: Generating Situated Robot Task Plans using Large Language
Models [68.57918965060787]
Large language models (LLMs) can be used to score potential next actions during task planning.
We present a programmatic LLM prompt structure that enables plan generation functional across situated environments.
arXiv Detail & Related papers (2022-09-22T20:29:49Z) - Towards Plug'n Play Task-Level Autonomy for Robotics Using POMDPs and
Generative Models [0.0]
We describe an approach for integrating robot skills into a working autonomous robot controller that schedules its skills to achieve a specified task.
Our Generative Skill Documentation Language (GSDL) makes code documentation compact and more expressive.
An abstraction mapping (AM) bridges the gap between low-level robot code and the abstract AI planning model.
arXiv Detail & Related papers (2022-07-20T07:27:47Z) - Can Foundation Models Perform Zero-Shot Task Specification For Robot
Manipulation? [54.442692221567796]
Task specification is critical for engagement of non-expert end-users and adoption of personalized robots.
A widely studied approach to task specification is through goals, using either compact state vectors or goal images from the same robot scene.
In this work, we explore alternate and more general forms of goal specification that are expected to be easier for humans to specify and use.
arXiv Detail & Related papers (2022-04-23T19:39:49Z) - Lifelong Robotic Reinforcement Learning by Retaining Experiences [61.79346922421323]
Many multi-task reinforcement learning efforts assume the robot can collect data from all tasks at all times.
In this work, we study a practical sequential multi-task RL problem motivated by the practical constraints of physical robotic systems.
We derive an approach that effectively leverages the data and policies learned for previous tasks to cumulatively grow the robot's skill-set.
arXiv Detail & Related papers (2021-09-19T18:00:51Z) - Learning Language-Conditioned Robot Behavior from Offline Data and
Crowd-Sourced Annotation [80.29069988090912]
We study the problem of learning a range of vision-based manipulation tasks from a large offline dataset of robot interaction.
We propose to leverage offline robot datasets with crowd-sourced natural language labels.
We find that our approach outperforms both goal-image specifications and language conditioned imitation techniques by more than 25%.
arXiv Detail & Related papers (2021-09-02T17:42:13Z) - DeComplex: Task planning from complex natural instructions by a
collocating robot [3.158346511479111]
It is not trivial to execute the human intended tasks as natural language expressions can have large linguistic variations.
Existing works assume either single task instruction is given to the robot at a time or there are multiple independent tasks in an instruction.
We propose a method to find the intended order of execution of multiple inter-dependent tasks given in natural language instruction.
arXiv Detail & Related papers (2020-08-23T18:10:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.