Option Discovery in the Absence of Rewards with Manifold Analysis

• URL: http://arxiv.org/abs/2003.05878v2
• Date: Wed, 19 Aug 2020 13:53:37 GMT
• Title: Option Discovery in the Absence of Rewards with Manifold Analysis
• Authors: Amitay Bar, Ronen Talmon and Ron Meir
• Abstract summary: We derive an algorithm that systematically discovers options without access to a specific reward or task assignment. As opposed to the common practice used in previous methods, our algorithm makes full use of the spectrum of the graph Laplacian.
• Score: 16.260113196512865
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Options have been shown to be an effective tool in reinforcement learning, facilitating improved exploration and learning. In this paper, we present an approach based on spectral graph theory and derive an algorithm that systematically discovers options without access to a specific reward or task assignment. As opposed to the common practice used in previous methods, our algorithm makes full use of the spectrum of the graph Laplacian. Incorporating modes associated with higher graph frequencies unravels domain subtleties, which are shown to be useful for option discovery. Using geometric and manifold-based analysis, we present a theoretical justification for the algorithm. In addition, we showcase its performance in several domains, demonstrating clear improvements compared to competing methods.

Related papers

• Consciousness-Inspired Spatio-Temporal Abstractions for Better Generalization in Reinforcement Learning [83.41487567765871]
  Skipper is a model-based reinforcement learning framework. It automatically generalizes the task given into smaller, more manageable subtasks. It enables sparse decision-making and focused abstractions on the relevant parts of the environment.
  arXiv  Detail & Related papers  (2023-09-30T02:25:18Z)
• Stochastic Graph Bandit Learning with Side-Observations [4.910658441596583]
  We propose an algorithm that adapts to both the underlying graph structures and reward gaps. To the best of our knowledge, our algorithm is the first to provide a gap-dependent upper bound in this setting.
  arXiv  Detail & Related papers  (2023-08-29T08:14:19Z)
• Gotta match 'em all: Solution diversification in graph matching matched filters [13.841897638543033]
  We present a novel approach for finding multiple noisily embedded template graphs in a very large background graph. Our method builds upon the graph-matching-matched-filter technique proposed in Sussman et al.
  arXiv  Detail & Related papers  (2023-08-25T15:53:30Z)
• The Galerkin method beats Graph-Based Approaches for Spectral Algorithms [3.5897534810405403]
  We break with the machine learning community and prove the statistical and computational superiority of the Galerkin method. We introduce implementation tricks to deal with differential operators in large dimensions with structured kernels. We extend on the core principles beyond our approach to apply them to non-linear spaces of functions, such as the ones parameterized by deep neural networks.
  arXiv  Detail & Related papers  (2023-06-01T14:38:54Z)
• Benchmarking Node Outlier Detection on Graphs [90.29966986023403]
  Graph outlier detection is an emerging but crucial machine learning task with numerous applications. We present the first comprehensive unsupervised node outlier detection benchmark for graphs called UNOD.
  arXiv  Detail & Related papers  (2022-06-21T01:46:38Z)
• Bayesian Graph Contrastive Learning [55.36652660268726]
  We propose a novel perspective of graph contrastive learning methods showing random augmentations leads to encoders. Our proposed method represents each node by a distribution in the latent space in contrast to existing techniques which embed each node to a deterministic vector. We show a considerable improvement in performance compared to existing state-of-the-art methods on several benchmark datasets.
  arXiv  Detail & Related papers  (2021-12-15T01:45:32Z)
• Graph-Embedded Subspace Support Vector Data Description [98.78559179013295]
  We propose a novel subspace learning framework for one-class classification. The proposed framework presents the problem in the form of graph embedding. We demonstrate improved performance against the baselines and the recently proposed subspace learning methods for one-class classification.
  arXiv  Detail & Related papers  (2021-04-29T14:30:48Z)
• Enhancing Balanced Graph Edge Partition with Effective Local Search [41.4257628524592]
  Graph partition is a key component to achieve workload balance and reduce job completion time in parallel graph processing systems. In this paper, we study local search algorithms for this problem to further improve the partition results from existing methods.
  arXiv  Detail & Related papers  (2020-12-17T08:58:06Z)
• Training Sensitivity in Graph Isomorphism Network [2.487445341407889]
  Graph neural network (GNN) is a popular tool to learn the lower-dimensional representation of a graph. This paper studies various alternative functions for a respective module using a diverse set of benchmark datasets.
  arXiv  Detail & Related papers  (2020-08-19T03:50:28Z)
• Online Dense Subgraph Discovery via Blurred-Graph Feedback [87.9850024070244]
  We introduce a novel learning problem for dense subgraph discovery. We first propose a edge-time algorithm that obtains a nearly-optimal solution with high probability. We then design a more scalable algorithm with a theoretical guarantee.
  arXiv  Detail & Related papers  (2020-06-24T11:37:33Z)
• Stochastic Flows and Geometric Optimization on the Orthogonal Group [52.50121190744979]
  We present a new class of geometrically-driven optimization algorithms on the orthogonal group $O(d)$. We show that our methods can be applied in various fields of machine learning including deep, convolutional and recurrent neural networks, reinforcement learning, flows and metric learning.
  arXiv  Detail & Related papers  (2020-03-30T15:37:50Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.