Symbolic Visual Reinforcement Learning: A Scalable Framework with Object-Level Abstraction and Differentiable Expression Search

• URL: http://arxiv.org/abs/2212.14849v1
• Date: Fri, 30 Dec 2022 17:50:54 GMT
• Title: Symbolic Visual Reinforcement Learning: A Scalable Framework with Object-Level Abstraction and Differentiable Expression Search
• Authors: Wenqing Zheng, S P Sharan, Zhiwen Fan, Kevin Wang, Yihan Xi, Zhangyang Wang
• Abstract summary: We propose DiffSES, a novel symbolic learning approach that discovers discrete symbolic policies. By using object-level abstractions instead of raw pixel-level inputs, DiffSES is able to leverage the simplicity and scalability advantages of symbolic expressions. Our experiments demonstrate that DiffSES is able to generate symbolic policies that are simpler and more scalable than state-of-the-art symbolic RL methods.
• Score: 63.3745291252038
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Learning efficient and interpretable policies has been a challenging task in reinforcement learning (RL), particularly in the visual RL setting with complex scenes. While neural networks have achieved competitive performance, the resulting policies are often over-parameterized black boxes that are difficult to interpret and deploy efficiently. More recent symbolic RL frameworks have shown that high-level domain-specific programming logic can be designed to handle both policy learning and symbolic planning. However, these approaches rely on coded primitives with little feature learning, and when applied to high-dimensional visual scenes, they can suffer from scalability issues and perform poorly when images have complex object interactions. To address these challenges, we propose \textit{Differentiable Symbolic Expression Search} (DiffSES), a novel symbolic learning approach that discovers discrete symbolic policies using partially differentiable optimization. By using object-level abstractions instead of raw pixel-level inputs, DiffSES is able to leverage the simplicity and scalability advantages of symbolic expressions, while also incorporating the strengths of neural networks for feature learning and optimization. Our experiments demonstrate that DiffSES is able to generate symbolic policies that are simpler and more and scalable than state-of-the-art symbolic RL methods, with a reduced amount of symbolic prior knowledge.

Related papers

• The Role of Foundation Models in Neuro-Symbolic Learning and Reasoning [54.56905063752427]
  Neuro-Symbolic AI (NeSy) holds promise to ensure the safe deployment of AI systems. Existing pipelines that train the neural and symbolic components sequentially require extensive labelling. New architecture, NeSyGPT, fine-tunes a vision-language foundation model to extract symbolic features from raw data.
  arXiv  Detail & Related papers  (2024-02-02T20:33:14Z)
• Efficient Symbolic Policy Learning with Differentiable Symbolic Expression [30.855457609733637]
  We propose an efficient gradient-based learning method that learns the symbolic policy from scratch in an end-to-end way. In addition, in contrast with previous symbolic policies which only work in single-task RL because of complexity, we expand ESPL on meta-RL to generate symbolic policies for unseen tasks.
  arXiv  Detail & Related papers  (2023-11-02T03:27:51Z)
• LOGICSEG: Parsing Visual Semantics with Neural Logic Learning and Reasoning [73.98142349171552]
  LOGICSEG is a holistic visual semantic that integrates neural inductive learning and logic reasoning with both rich data and symbolic knowledge. During fuzzy logic-based continuous relaxation, logical formulae are grounded onto data and neural computational graphs, hence enabling logic-induced network training. These designs together make LOGICSEG a general and compact neural-logic machine that is readily integrated into existing segmentation models.
  arXiv  Detail & Related papers  (2023-09-24T05:43:19Z)
• Goal Space Abstraction in Hierarchical Reinforcement Learning via Set-Based Reachability Analysis [0.5409704301731713]
  We introduce a Feudal HRL algorithm that concurrently learns both the goal representation and a hierarchical policy. We evaluate our approach on complex navigation tasks, showing the learned representation is interpretable, transferrable and results in data efficient learning.
  arXiv  Detail & Related papers  (2023-09-14T12:39:26Z)
• Goal Space Abstraction in Hierarchical Reinforcement Learning via Reachability Analysis [0.0]
  We propose a developmental mechanism for subgoal discovery via an emergent representation that abstracts (i.e., groups together) sets of environment states. We create a HRL algorithm that gradually learns this representation along with the policies and evaluate it on navigation tasks to show the learned representation is interpretable and results in data efficiency.
  arXiv  Detail & Related papers  (2023-09-12T06:53:11Z)
• Improving Deep Representation Learning via Auxiliary Learnable Target Coding [69.79343510578877]
  This paper introduces a novel learnable target coding as an auxiliary regularization of deep representation learning. Specifically, a margin-based triplet loss and a correlation consistency loss on the proposed target codes are designed to encourage more discriminative representations.
  arXiv  Detail & Related papers  (2023-05-30T01:38:54Z)
• Symbolic Synthesis of Neural Networks [0.0]
  I present Graph-basedally Synthesized Neural Networks (GSSNNs) GSSNNs are a form of neural network whose topology and parameters are informed by the output of a symbolic program. I demonstrate that by developing symbolic abstractions at a population level, I can elicit reliable patterns of improved generalization with small quantities of data known to contain local and discrete features.
  arXiv  Detail & Related papers  (2023-03-06T18:13:14Z)
• Emergent Symbols through Binding in External Memory [2.3562267625320352]
  We introduce the Emergent Symbol Binding Network (ESBN), a recurrent network augmented with an external memory. This binding mechanism allows symbol-like representations to emerge through the learning process without the need to explicitly incorporate symbol-processing machinery. Across a series of tasks, we show that this architecture displays nearly perfect generalization of learned rules to novel entities.
  arXiv  Detail & Related papers  (2020-12-29T04:28:32Z)
• Synbols: Probing Learning Algorithms with Synthetic Datasets [112.45883250213272]
  Synbols is a tool for rapidly generating new datasets with a rich composition of latent features rendered in low resolution images. Our tool's high-level interface provides a language for rapidly generating new distributions on the latent features. To showcase the versatility of Synbols, we use it to dissect the limitations and flaws in standard learning algorithms in various learning setups.
  arXiv  Detail & Related papers  (2020-09-14T13:03:27Z)
• Symbol Spotting on Digital Architectural Floor Plans Using a Deep Learning-based Framework [76.70609932823149]
  This paper focuses on symbol spotting on real-world digital architectural floor plans with a deep learning (DL)-based framework. We propose a training strategy based on tiles, avoiding many issues particular to DL-based object detection networks. Experiments on real-world floor plans demonstrate that our method successfully detects architectural symbols with low intra-class similarity and of variable graphical complexity.
  arXiv  Detail & Related papers  (2020-06-01T03:16:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.