Human-guided Collaborative Problem Solving: A Natural Language based Framework

• URL: http://arxiv.org/abs/2207.09566v1
• Date: Tue, 19 Jul 2022 21:52:37 GMT
• Title: Human-guided Collaborative Problem Solving: A Natural Language based Framework
• Authors: Harsha Kokel, Mayukh Das, Rakibul Islam, Julia Bonn, Jon Cai, Soham Dan, Anjali Narayan-Chen, Prashant Jayannavar, Janardhan Rao Doppa, Julia Hockenmaier, Sriraam Natarajan, Martha Palmer, Dan Roth
• Abstract summary: Our framework consists of three components -- a natural language engine that parses the language utterances to a formal representation and vice-versa. We illustrate the ability of this framework to address the key challenges of collaborative problem solving by demonstrating it on a collaborative building task in a Minecraft-based blocksworld domain.
• Score: 74.27063862727849
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We consider the problem of human-machine collaborative problem solving as a planning task coupled with natural language communication. Our framework consists of three components -- a natural language engine that parses the language utterances to a formal representation and vice-versa, a concept learner that induces generalized concepts for plans based on limited interactions with the user, and an HTN planner that solves the task based on human interaction. We illustrate the ability of this framework to address the key challenges of collaborative problem solving by demonstrating it on a collaborative building task in a Minecraft-based blocksworld domain. The accompanied demo video is available at https://youtu.be/q1pWe4aahF0.

Related papers

• Your Co-Workers Matter: Evaluating Collaborative Capabilities of Language Models in Blocks World [13.005764902339523]
  We design a blocks-world environment where two agents, each having unique goals and skills, build a target structure together. To complete the goals, they can act in the world and communicate in natural language. We adopt chain-of-thought prompts that include intermediate reasoning steps to model the partner's state and identify and correct execution errors.
  arXiv  Detail & Related papers  (2024-03-30T04:48:38Z)
• GOMA: Proactive Embodied Cooperative Communication via Goal-Oriented Mental Alignment [72.96949760114575]
  We propose a novel cooperative communication framework, Goal-Oriented Mental Alignment (GOMA) GOMA formulates verbal communication as a planning problem that minimizes the misalignment between parts of agents' mental states that are relevant to the goals. We evaluate our approach against strong baselines in two challenging environments, Overcooked (a multiplayer game) and VirtualHome (a household simulator)
  arXiv  Detail & Related papers  (2024-03-17T03:52:52Z)
• ChatGPT and Other Large Language Models as Evolutionary Engines for Online Interactive Collaborative Game Design [0.0]
  We present a collaborative game design framework that combines interactive evolution and large language models. In our framework, the process starts with a brief and a set of candidate designs, either generated using a language model or proposed by the users. Next, users collaborate on the design process by providing feedback to an interactive genetic algorithm that selects, recombines, and mutates the most promising designs.
  arXiv  Detail & Related papers  (2023-02-09T15:44:43Z)
• Coarse-to-Fine: Hierarchical Multi-task Learning for Natural Language Understanding [51.31622274823167]
  We propose a hierarchical framework with a coarse-to-fine paradigm, with the bottom level shared to all the tasks, the mid-level divided to different groups, and the top-level assigned to each of the tasks. This allows our model to learn basic language properties from all tasks, boost performance on relevant tasks, and reduce the negative impact from irrelevant tasks.
  arXiv  Detail & Related papers  (2022-08-19T02:46:20Z)
• IGLU 2022: Interactive Grounded Language Understanding in a Collaborative Environment at NeurIPS 2022 [63.07251290802841]
  We propose IGLU: Interactive Grounded Language Understanding in a Collaborative Environment. The primary goal of the competition is to approach the problem of how to develop interactive embodied agents. This research challenge is naturally related, but not limited, to two fields of study that are highly relevant to the NeurIPS community.
  arXiv  Detail & Related papers  (2022-05-27T06:12:48Z)
• Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 2021 [58.196738777207315]
  We propose emphIGLU: Interactive Grounded Language Understanding in a Collaborative Environment. The primary goal of the competition is to approach the problem of how to build interactive agents that learn to solve a task while provided with grounded natural language instructions in a collaborative environment.
  arXiv  Detail & Related papers  (2022-05-05T01:20:09Z)
• Natural Language for Human-Robot Collaboration: Problems Beyond Language Grounding [10.227242085922613]
  We identify several aspects of language processing that are not commonly studied in this context. These include location, planning, and generation. We suggest evaluations for each task, offer baselines for simple methods, and close by discussing challenges and opportunities in studying language for collaboration.
  arXiv  Detail & Related papers  (2021-10-09T03:24:38Z)
• MindCraft: Theory of Mind Modeling for Situated Dialogue in Collaborative Tasks [2.5725755841426623]
  Theory of mind plays an important role in maintaining common ground during human collaboration and communication. We introduce a fine-grained dataset of collaborative tasks performed by pairs of human subjects in the 3D virtual blocks world of Minecraft. It provides information that captures partners' beliefs of the world and of each other as an interaction unfolds.
  arXiv  Detail & Related papers  (2021-09-13T19:26:19Z)
• Experience Grounds Language [185.73483760454454]
  Language understanding research is held back by a failure to relate language to the physical world it describes and to the social interactions it facilitates. Despite the incredible effectiveness of language processing models to tackle tasks after being trained on text alone, successful linguistic communication relies on a shared experience of the world.
  arXiv  Detail & Related papers  (2020-04-21T16:56:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.