Learning with Language-Guided State Abstractions

• URL: http://arxiv.org/abs/2402.18759v2
• Date: Wed, 6 Mar 2024 15:53:46 GMT
• Title: Learning with Language-Guided State Abstractions
• Authors: Andi Peng, Ilia Sucholutsky, Belinda Z. Li, Theodore R. Sumers, Thomas L. Griffiths, Jacob Andreas, Julie A. Shah
• Abstract summary: Generalizable policy learning in high-dimensional observation spaces is facilitated by well-designed state representations. Our method, LGA, uses a combination of natural language supervision and background knowledge from language models to automatically build state representations tailored to unseen tasks. Experiments on simulated robotic tasks show that LGA yields state abstractions similar to those designed by humans, but in a fraction of the time.
• Score: 58.199148890064826
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: We describe a framework for using natural language to design state abstractions for imitation learning. Generalizable policy learning in high-dimensional observation spaces is facilitated by well-designed state representations, which can surface important features of an environment and hide irrelevant ones. These state representations are typically manually specified, or derived from other labor-intensive labeling procedures. Our method, LGA (language-guided abstraction), uses a combination of natural language supervision and background knowledge from language models (LMs) to automatically build state representations tailored to unseen tasks. In LGA, a user first provides a (possibly incomplete) description of a target task in natural language; next, a pre-trained LM translates this task description into a state abstraction function that masks out irrelevant features; finally, an imitation policy is trained using a small number of demonstrations and LGA-generated abstract states. Experiments on simulated robotic tasks show that LGA yields state abstractions similar to those designed by humans, but in a fraction of the time, and that these abstractions improve generalization and robustness in the presence of spurious correlations and ambiguous specifications. We illustrate the utility of the learned abstractions on mobile manipulation tasks with a Spot robot.

Related papers

• VisualPredicator: Learning Abstract World Models with Neuro-Symbolic Predicates for Robot Planning [86.59849798539312]
  We present Neuro-Symbolic Predicates, a first-order abstraction language that combines the strengths of symbolic and neural knowledge representations. We show that our approach offers better sample complexity, stronger out-of-distribution generalization, and improved interpretability.
  arXiv  Detail & Related papers  (2024-10-30T16:11:05Z)
• Learning Planning Abstractions from Language [28.855381137615275]
  This paper presents a framework for learning state and action abstractions in sequential decision-making domains. Our framework, planning abstraction from language (PARL), utilizes language-annotated demonstrations to automatically discover a symbolic and abstract action space.
  arXiv  Detail & Related papers  (2024-05-06T21:24:22Z)
• Preference-Conditioned Language-Guided Abstraction [24.626805570296064]
  We observe that how humans behave reveals how they see the world. In this work, we propose using language models (LMs) to query for those preferences directly given knowledge that a change in behavior has occurred. We demonstrate our framework's ability to construct effective preference-conditioned abstractions in simulated experiments, a user study, and on a real Spot robot performing mobile manipulation tasks.
  arXiv  Detail & Related papers  (2024-02-05T15:12:15Z)
• Object-Centric Instruction Augmentation for Robotic Manipulation [29.491990994901666]
  We introduce the textitObject-Centric Instruction Augmentation (OCI) framework to augment highly semantic and information-dense language instruction with position cues. We utilize a Multi-modal Large Language Model (MLLM) to weave knowledge of object locations into natural language instruction. We demonstrate that robotic manipulator imitation policies trained with our enhanced instructions outperform those relying solely on traditional language instructions.
  arXiv  Detail & Related papers  (2024-01-05T13:54:45Z)
• AbsPyramid: Benchmarking the Abstraction Ability of Language Models with a Unified Entailment Graph [62.685920585838616]
  abstraction ability is essential in human intelligence, which remains under-explored in language models. We present AbsPyramid, a unified entailment graph of 221K textual descriptions of abstraction knowledge.
  arXiv  Detail & Related papers  (2023-11-15T18:11:23Z)
• ARNOLD: A Benchmark for Language-Grounded Task Learning With Continuous States in Realistic 3D Scenes [72.83187997344406]
  ARNOLD is a benchmark that evaluates language-grounded task learning with continuous states in realistic 3D scenes. ARNOLD is comprised of 8 language-conditioned tasks that involve understanding object states and learning policies for continuous goals.
  arXiv  Detail & Related papers  (2023-04-09T21:42:57Z)
• Semantic Exploration from Language Abstractions and Pretrained Representations [23.02024937564099]
  Effective exploration is a challenge in reinforcement learning (RL) We define novelty using semantically meaningful state abstractions. We evaluate vision-language representations, pretrained on natural image captioning datasets.
  arXiv  Detail & Related papers  (2022-04-08T17:08:00Z)
• LISA: Learning Interpretable Skill Abstractions from Language [85.20587800593293]
  We propose a hierarchical imitation learning framework that can learn diverse, interpretable skills from language-conditioned demonstrations. Our method demonstrates a more natural way to condition on language in sequential decision-making problems.
  arXiv  Detail & Related papers  (2022-02-28T19:43:24Z)
• A Persistent Spatial Semantic Representation for High-level Natural Language Instruction Execution [54.385344986265714]
  We propose a persistent spatial semantic representation method to bridge the gap between language and robot actions. We evaluate our approach on the ALFRED benchmark and achieve state-of-the-art results, despite completely avoiding the commonly used step-by-step instructions.
  arXiv  Detail & Related papers  (2021-07-12T17:47:19Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.