LISA: Learning Interpretable Skill Abstractions from Language

• URL: http://arxiv.org/abs/2203.00054v1
• Date: Mon, 28 Feb 2022 19:43:24 GMT
• Title: LISA: Learning Interpretable Skill Abstractions from Language
• Authors: Divyansh Garg, Skanda Vaidyanath, Kuno Kim, Jiaming Song, Stefano Ermon
• Abstract summary: We propose a hierarchical imitation learning framework that can learn diverse, interpretable skills from language-conditioned demonstrations. Our method demonstrates a more natural way to condition on language in sequential decision-making problems.
• Score: 85.20587800593293
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning policies that effectually utilize language instructions in complex, multi-task environments is an important problem in imitation learning. While it is possible to condition on the entire language instruction directly, such an approach could suffer from generalization issues. To encode complex instructions into skills that can generalize to unseen instructions, we propose Learning Interpretable Skill Abstractions (LISA), a hierarchical imitation learning framework that can learn diverse, interpretable skills from language-conditioned demonstrations. LISA uses vector quantization to learn discrete skill codes that are highly correlated with language instructions and the behavior of the learned policy. In navigation and robotic manipulation environments, LISA is able to outperform a strong non-hierarchical baseline in the low data regime and compose learned skills to solve tasks containing unseen long-range instructions. Our method demonstrates a more natural way to condition on language in sequential decision-making problems and achieve interpretable and controllable behavior with the learned skills.

Related papers

• Teaching Embodied Reinforcement Learning Agents: Informativeness and Diversity of Language Use [16.425032085699698]
  It is desirable for embodied agents to have the ability to leverage human language to gain explicit or implicit knowledge for learning tasks. It's not clear how to incorporate rich language use to facilitate task learning. This paper studies different types of language inputs in facilitating reinforcement learning.
  arXiv  Detail & Related papers  (2024-10-31T17:59:52Z)
• Large Language Models are Interpretable Learners [53.56735770834617]
  In this paper, we show a combination of Large Language Models (LLMs) and symbolic programs can bridge the gap between expressiveness and interpretability. The pretrained LLM with natural language prompts provides a massive set of interpretable modules that can transform raw input into natural language concepts. As the knowledge learned by LSP is a combination of natural language descriptions and symbolic rules, it is easily transferable to humans (interpretable) and other LLMs.
  arXiv  Detail & Related papers  (2024-06-25T02:18:15Z)
• Interpretable Robotic Manipulation from Language [11.207620790833271]
  We introduce an explainable behavior cloning agent, named Ex-PERACT, specifically designed for manipulation tasks. At the top level, the model is tasked with learning a discrete skill code, while at the bottom level, the policy network translates the problem into a voxelized grid and maps the discretized actions to voxel grids. We evaluate our method across eight challenging manipulation tasks utilizing the RLBench benchmark, demonstrating that Ex-PERACT not only achieves competitive policy performance but also effectively bridges the gap between human instructions and machine execution in complex environments.
  arXiv  Detail & Related papers  (2024-05-27T11:02:21Z)
• Learning with Language-Guided State Abstractions [58.199148890064826]
  Generalizable policy learning in high-dimensional observation spaces is facilitated by well-designed state representations. Our method, LGA, uses a combination of natural language supervision and background knowledge from language models to automatically build state representations tailored to unseen tasks. Experiments on simulated robotic tasks show that LGA yields state abstractions similar to those designed by humans, but in a fraction of the time.
  arXiv  Detail & Related papers  (2024-02-28T23:57:04Z)
• Rethinking Mutual Information for Language Conditioned Skill Discovery on Imitation Learning [36.624923972563415]
  We propose an end-to-end imitation learning approach known as Language Conditioned Skill Discovery (LCSD) We utilize vector quantization to learn discrete latent skills and leverage skill sequences of trajectories to reconstruct high-level semantic instructions. Our approach exhibits enhanced generalization capabilities towards unseen tasks, improved skill interpretability, and notably higher rates of task completion success.
  arXiv  Detail & Related papers  (2024-02-27T13:53:52Z)
• VAL: Interactive Task Learning with GPT Dialog Parsing [2.6207405455197827]
  Large language models (LLMs) are resistant to brittleness but are not interpretable and cannot learn incrementally. We present VAL, an ITL system with a new philosophy for LLM/symbolic integration. We studied users' interactions with VAL in a video game setting, finding that most users could successfully teach VAL using language they felt was natural.
  arXiv  Detail & Related papers  (2023-10-02T20:45:41Z)
• InstructAlign: High-and-Low Resource Language Alignment via Continual Crosslingual Instruction Tuning [66.31509106146605]
  Large language models (LLMs) that are tuned with instructions have demonstrated remarkable capabilities in various tasks and languages. However, their ability to generalize to underrepresented languages is limited due to the scarcity of available data. We propose InstructAlign which uses continual crosslingual instruction tuning to enable LLMs to align new unseen languages with previously learned high-resource languages.
  arXiv  Detail & Related papers  (2023-05-23T02:51:34Z)
• PADL: Language-Directed Physics-Based Character Control [66.517142635815]
  We present PADL, which allows users to issue natural language commands for specifying high-level tasks and low-level skills that a character should perform. We show that our framework can be applied to effectively direct a simulated humanoid character to perform a diverse array of complex motor skills.
  arXiv  Detail & Related papers  (2023-01-31T18:59:22Z)
• Ask Your Humans: Using Human Instructions to Improve Generalization in Reinforcement Learning [32.82030512053361]
  We propose the use of step-by-step human demonstrations in the form of natural language instructions and action trajectories. We find that human demonstrations help solve the most complex tasks. We also find that incorporating natural language allows the model to generalize to unseen tasks in a zero-shot setting.
  arXiv  Detail & Related papers  (2020-11-01T14:39:46Z)
• Learning Adaptive Language Interfaces through Decomposition [89.21937539950966]
  We introduce a neural semantic parsing system that learns new high-level abstractions through decomposition. Users interactively teach the system by breaking down high-level utterances describing novel behavior into low-level steps.
  arXiv  Detail & Related papers  (2020-10-11T08:27:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.