Learning Symbolic Rules over Abstract Meaning Representations for Textual Reinforcement Learning

• URL: http://arxiv.org/abs/2307.02689v1
• Date: Wed, 5 Jul 2023 23:21:05 GMT
• Title: Learning Symbolic Rules over Abstract Meaning Representations for Textual Reinforcement Learning
• Authors: Subhajit Chaudhury, Sarathkrishna Swaminathan, Daiki Kimura, Prithviraj Sen, Keerthiram Murugesan, Rosario Uceda-Sosa, Michiaki Tatsubori, Achille Fokoue, Pavan Kapanipathi, Asim Munawar and Alexander Gray
• Abstract summary: We propose a modular, NEuroSymbolic Textual Agent (NESTA) that combines a generic semantic generalization with a rule induction system to learn interpretable rules as policies. Our experiments show that the proposed NESTA method outperforms deep reinforcement learning-based techniques by achieving better to unseen test games and learning from fewer training interactions.
• Score: 63.148199057487226
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Text-based reinforcement learning agents have predominantly been neural network-based models with embeddings-based representation, learning uninterpretable policies that often do not generalize well to unseen games. On the other hand, neuro-symbolic methods, specifically those that leverage an intermediate formal representation, are gaining significant attention in language understanding tasks. This is because of their advantages ranging from inherent interpretability, the lesser requirement of training data, and being generalizable in scenarios with unseen data. Therefore, in this paper, we propose a modular, NEuro-Symbolic Textual Agent (NESTA) that combines a generic semantic parser with a rule induction system to learn abstract interpretable rules as policies. Our experiments on established text-based game benchmarks show that the proposed NESTA method outperforms deep reinforcement learning-based techniques by achieving better generalization to unseen test games and learning from fewer training interactions.

Related papers

• Pre-trained Text-to-Image Diffusion Models Are Versatile Representation Learners for Control [73.6361029556484]
  Embodied AI agents require a fine-grained understanding of the physical world mediated through visual and language inputs. We consider pre-trained text-to-image diffusion models, which are explicitly optimized to generate images from text prompts. We show that Stable Control Representations enable learning policies that exhibit state-of-the-art performance on OVMM, a difficult open-vocabulary navigation benchmark.
  arXiv  Detail & Related papers  (2024-05-09T15:39:54Z)
• On the Effects of Fine-tuning Language Models for Text-Based Reinforcement Learning [19.057241328691077]
  We show that rich semantic understanding leads to efficient training of text-based RL agents. We describe the occurrence of semantic degeneration as a consequence of inappropriate fine-tuning of language models.
  arXiv  Detail & Related papers  (2024-04-15T23:05:57Z)
• Pre-Training to Learn in Context [138.0745138788142]
  The ability of in-context learning is not fully exploited because language models are not explicitly trained to learn in context. We propose PICL (Pre-training for In-Context Learning), a framework to enhance the language models' in-context learning ability. Our experiments show that PICL is more effective and task-generalizable than a range of baselines, outperforming larger language models with nearly 4x parameters.
  arXiv  Detail & Related papers  (2023-05-16T03:38:06Z)
• Neuro-Symbolic Reinforcement Learning with First-Order Logic [63.003353499732434]
  We propose a novel RL method for text-based games with a recent neuro-symbolic framework called Logical Neural Network. Our experimental results show RL training with the proposed method converges significantly faster than other state-of-the-art neuro-symbolic methods in a TextWorld benchmark.
  arXiv  Detail & Related papers  (2021-10-21T08:21:49Z)
• Dependency Induction Through the Lens of Visual Perception [81.91502968815746]
  We propose an unsupervised grammar induction model that leverages word concreteness and a structural vision-based to jointly learn constituency-structure and dependency-structure grammars. Our experiments show that the proposed extension outperforms the current state-of-the-art visually grounded models in constituency parsing even with a smaller grammar size.
  arXiv  Detail & Related papers  (2021-09-20T18:40:37Z)
• Continual Learning for Text Classification with Information Disentanglement Based Regularization [18.258948837964724]
  We propose an information disentanglement based regularization method for continual learning on text classification. Experiments conducted on large-scale benchmarks demonstrate the effectiveness of our method.
  arXiv  Detail & Related papers  (2021-04-12T14:17:43Z)
• Deep Reinforcement Learning with Stacked Hierarchical Attention for Text-based Games [64.11746320061965]
  We study reinforcement learning for text-based games, which are interactive simulations in the context of natural language. We aim to conduct explicit reasoning with knowledge graphs for decision making, so that the actions of an agent are generated and supported by an interpretable inference procedure. We extensively evaluate our method on a number of man-made benchmark games, and the experimental results demonstrate that our method performs better than existing text-based agents.
  arXiv  Detail & Related papers  (2020-10-22T12:40:22Z)
• Systematic Generalization on gSCAN with Language Conditioned Embedding [19.39687991647301]
  Systematic Generalization refers to a learning algorithm's ability to extrapolate learned behavior to unseen situations. We propose a novel method that learns objects' contextualized embeddings with dynamic message passing conditioned on the input natural language.
  arXiv  Detail & Related papers  (2020-09-11T17:35:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.