Enabling human-like task identification from natural conversation
- URL: http://arxiv.org/abs/2008.10073v2
- Date: Sat, 29 Aug 2020 04:54:20 GMT
- Title: Enabling human-like task identification from natural conversation
- Authors: Pradip Pramanick, Chayan Sarkar, Balamuralidhar P, Ajay Kattepur,
Indrajit Bhattacharya, Arpan Pal
- Abstract summary: We provide a non-trivial method to combine an NLP engine and a planner such that a robot can successfully identify tasks and all the relevant parameters and generate an accurate plan for the task.
This work makes a significant stride towards enabling a human-like task understanding capability in a robot.
- Score: 7.00597813134145
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: A robot as a coworker or a cohabitant is becoming mainstream day-by-day with
the development of low-cost sophisticated hardware. However, an accompanying
software stack that can aid the usability of the robotic hardware remains the
bottleneck of the process, especially if the robot is not dedicated to a single
job. Programming a multi-purpose robot requires an on the fly mission
scheduling capability that involves task identification and plan generation.
The problem dimension increases if the robot accepts tasks from a human in
natural language. Though recent advances in NLP and planner development can
solve a variety of complex problems, their amalgamation for a dynamic robotic
task handler is used in a limited scope. Specifically, the problem of
formulating a planning problem from natural language instructions is not
studied in details. In this work, we provide a non-trivial method to combine an
NLP engine and a planner such that a robot can successfully identify tasks and
all the relevant parameters and generate an accurate plan for the task.
Additionally, some mechanism is required to resolve the ambiguity or missing
pieces of information in natural language instruction. Thus, we also develop a
dialogue strategy that aims to gather additional information with minimal
question-answer iterations and only when it is necessary. This work makes a
significant stride towards enabling a human-like task understanding capability
in a robot.
Related papers
- $π_0$: A Vision-Language-Action Flow Model for General Robot Control [77.32743739202543]
We propose a novel flow matching architecture built on top of a pre-trained vision-language model (VLM) to inherit Internet-scale semantic knowledge.
We evaluate our model in terms of its ability to perform tasks in zero shot after pre-training, follow language instructions from people, and its ability to acquire new skills via fine-tuning.
arXiv Detail & Related papers (2024-10-31T17:22:30Z) - COHERENT: Collaboration of Heterogeneous Multi-Robot System with Large Language Models [49.24666980374751]
COHERENT is a novel LLM-based task planning framework for collaboration of heterogeneous multi-robot systems.
A Proposal-Execution-Feedback-Adjustment mechanism is designed to decompose and assign actions for individual robots.
The experimental results show that our work surpasses the previous methods by a large margin in terms of success rate and execution efficiency.
arXiv Detail & Related papers (2024-09-23T15:53:41Z) - Autonomous Behavior Planning For Humanoid Loco-manipulation Through Grounded Language Model [6.9268843428933025]
Large language models (LLMs) have demonstrated powerful planning and reasoning capabilities for comprehension and processing of semantic information.
We propose a novel language-model based framework that enables robots to autonomously plan behaviors and low-level execution under given textual instructions.
arXiv Detail & Related papers (2024-08-15T17:33:32Z) - Automated Process Planning Based on a Semantic Capability Model and SMT [50.76251195257306]
In research of manufacturing systems and autonomous robots, the term capability is used for a machine-interpretable specification of a system function.
We present an approach that combines these two topics: starting from a semantic capability model, an AI planning problem is automatically generated.
arXiv Detail & Related papers (2023-12-14T10:37:34Z) - Logic programming for deliberative robotic task planning [2.610470075814367]
We present a survey on recent advances in the application of logic programming to the problem of task planning.
We analyze different planners and their suitability for specific robotic applications, based on expressivity in domain representation, computational efficiency and software implementation.
arXiv Detail & Related papers (2023-01-18T14:11:55Z) - ProgPrompt: Generating Situated Robot Task Plans using Large Language
Models [68.57918965060787]
Large language models (LLMs) can be used to score potential next actions during task planning.
We present a programmatic LLM prompt structure that enables plan generation functional across situated environments.
arXiv Detail & Related papers (2022-09-22T20:29:49Z) - Towards Plug'n Play Task-Level Autonomy for Robotics Using POMDPs and
Generative Models [0.0]
We describe an approach for integrating robot skills into a working autonomous robot controller that schedules its skills to achieve a specified task.
Our Generative Skill Documentation Language (GSDL) makes code documentation compact and more expressive.
An abstraction mapping (AM) bridges the gap between low-level robot code and the abstract AI planning model.
arXiv Detail & Related papers (2022-07-20T07:27:47Z) - Lifelong Robotic Reinforcement Learning by Retaining Experiences [61.79346922421323]
Many multi-task reinforcement learning efforts assume the robot can collect data from all tasks at all times.
In this work, we study a practical sequential multi-task RL problem motivated by the practical constraints of physical robotic systems.
We derive an approach that effectively leverages the data and policies learned for previous tasks to cumulatively grow the robot's skill-set.
arXiv Detail & Related papers (2021-09-19T18:00:51Z) - Learning Language-Conditioned Robot Behavior from Offline Data and
Crowd-Sourced Annotation [80.29069988090912]
We study the problem of learning a range of vision-based manipulation tasks from a large offline dataset of robot interaction.
We propose to leverage offline robot datasets with crowd-sourced natural language labels.
We find that our approach outperforms both goal-image specifications and language conditioned imitation techniques by more than 25%.
arXiv Detail & Related papers (2021-09-02T17:42:13Z) - DeComplex: Task planning from complex natural instructions by a
collocating robot [3.158346511479111]
It is not trivial to execute the human intended tasks as natural language expressions can have large linguistic variations.
Existing works assume either single task instruction is given to the robot at a time or there are multiple independent tasks in an instruction.
We propose a method to find the intended order of execution of multiple inter-dependent tasks given in natural language instruction.
arXiv Detail & Related papers (2020-08-23T18:10:24Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.