Grounding Language to Autonomously-Acquired Skills via Goal Generation

• URL: http://arxiv.org/abs/2006.07185v3
• Date: Mon, 25 Jan 2021 15:47:16 GMT
• Title: Grounding Language to Autonomously-Acquired Skills via Goal Generation
• Authors: Ahmed Akakzia, C\'edric Colas, Pierre-Yves Oudeyer, Mohamed Chetouani, Olivier Sigaud
• Abstract summary: We propose a new conceptual approach to language-conditioned RL: the Language-Goal-Behavior architecture (LGB) LGB decouples skill learning and language grounding via an intermediate semantic representation of the world. We present DECSTR, an intrinsically motivated learning agent endowed with an innate semantic representation describing spatial relations between physical objects.
• Score: 23.327749767424567
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We are interested in the autonomous acquisition of repertoires of skills. Language-conditioned reinforcement learning (LC-RL) approaches are great tools in this quest, as they allow to express abstract goals as sets of constraints on the states. However, most LC-RL agents are not autonomous and cannot learn without external instructions and feedback. Besides, their direct language condition cannot account for the goal-directed behavior of pre-verbal infants and strongly limits the expression of behavioral diversity for a given language input. To resolve these issues, we propose a new conceptual approach to language-conditioned RL: the Language-Goal-Behavior architecture (LGB). LGB decouples skill learning and language grounding via an intermediate semantic representation of the world. To showcase the properties of LGB, we present a specific implementation called DECSTR. DECSTR is an intrinsically motivated learning agent endowed with an innate semantic representation describing spatial relations between physical objects. In a first stage (G -> B), it freely explores its environment and targets self-generated semantic configurations. In a second stage (L -> G), it trains a language-conditioned goal generator to generate semantic goals that match the constraints expressed in language-based inputs. We showcase the additional properties of LGB w.r.t. both an end-to-end LC-RL approach and a similar approach leveraging non-semantic, continuous intermediate representations. Intermediate semantic representations help satisfy language commands in a diversity of ways, enable strategy switching after a failure and facilitate language grounding.

Related papers

• Lens: Rethinking Multilingual Enhancement for Large Language Models [70.85065197789639]
  Lens is a novel approach to enhance multilingual capabilities of large language models (LLMs) It operates by manipulating the hidden representations within the language-agnostic and language-specific subspaces from top layers of LLMs. It achieves superior results with much fewer computational resources compared to existing post-training approaches.
  arXiv  Detail & Related papers  (2024-10-06T08:51:30Z)
• Learning with Language-Guided State Abstractions [58.199148890064826]
  Generalizable policy learning in high-dimensional observation spaces is facilitated by well-designed state representations. Our method, LGA, uses a combination of natural language supervision and background knowledge from language models to automatically build state representations tailored to unseen tasks. Experiments on simulated robotic tasks show that LGA yields state abstractions similar to those designed by humans, but in a fraction of the time.
  arXiv  Detail & Related papers  (2024-02-28T23:57:04Z)
• Are Structural Concepts Universal in Transformer Language Models? Towards Interpretable Cross-Lingual Generalization [27.368684663279463]
  We investigate the potential for explicitly aligning conceptual correspondence between languages to enhance cross-lingual generalization. Using the syntactic aspect of language as a testbed, our analyses of 43 languages reveal a high degree of alignability. We propose a meta-learning-based method to learn to align conceptual spaces of different languages.
  arXiv  Detail & Related papers  (2023-10-19T14:50:51Z)
• Simple Embodied Language Learning as a Byproduct of Meta-Reinforcement Learning [56.07190845063208]
  We ask: can embodied reinforcement learning (RL) agents indirectly learn language from non-language tasks? We design an office navigation environment, where the agent's goal is to find a particular office, and office locations differ in different buildings (i.e., tasks) We find RL agents indeed are able to indirectly learn language. Agents trained with current meta-RL algorithms successfully generalize to reading floor plans with held-out layouts and language phrases.
  arXiv  Detail & Related papers  (2023-06-14T09:48:48Z)
• GL-CLeF: A Global-Local Contrastive Learning Framework for Cross-lingual Spoken Language Understanding [74.39024160277809]
  We present Global--Local Contrastive Learning Framework (GL-CLeF) to address this shortcoming. Specifically, we employ contrastive learning, leveraging bilingual dictionaries to construct multilingual views of the same utterance. GL-CLeF achieves the best performance and successfully pulls representations of similar sentences across languages closer.
  arXiv  Detail & Related papers  (2022-04-18T13:56:58Z)
• Integrating Language Guidance into Vision-based Deep Metric Learning [78.18860829585182]
  We propose to learn metric spaces which encode semantic similarities as embedding space. These spaces should be transferable to classes beyond those seen during training. This causes learned embedding spaces to encode incomplete semantic context and misrepresent the semantic relation between classes.
  arXiv  Detail & Related papers  (2022-03-16T11:06:50Z)
• Pre-Trained Language Models for Interactive Decision-Making [72.77825666035203]
  We describe a framework for imitation learning in which goals and observations are represented as a sequence of embeddings. We demonstrate that this framework enables effective generalization across different environments. For test tasks involving novel goals or novel scenes, initializing policies with language models improves task completion rates by 43.6%.
  arXiv  Detail & Related papers  (2022-02-03T18:55:52Z)
• ReferentialGym: A Nomenclature and Framework for Language Emergence & Grounding in (Visual) Referential Games [0.30458514384586394]
  Natural languages are powerful tools wielded by human beings to communicate information and co-operate towards common goals. computational linguists have been researching the emergence of in artificial languages induced by language games. The AI community has started to investigate language emergence and grounding working towards better human-machine interfaces.
  arXiv  Detail & Related papers  (2020-12-17T10:22:15Z)
• Inverse Reinforcement Learning with Natural Language Goals [8.972202854038382]
  We propose a novel inverse reinforcement learning algorithm to learn a language-conditioned policy and reward function. Our algorithm outperforms multiple baselines by a large margin on a vision-based natural language instruction following dataset.
  arXiv  Detail & Related papers  (2020-08-16T14:43:49Z)
• Language-Conditioned Goal Generation: a New Approach to Language Grounding for RL [23.327749767424567]
  In the real world, linguistic agents are also embodied agents: they perceive and act in the physical world. This paper proposes using language to condition goal generators. Given any goal-conditioned policy, one could train a language-conditioned goal generator to generate language-agnostic goals for the agent.
  arXiv  Detail & Related papers  (2020-06-12T09:54:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.