Autonomous sPOMDP Environment Modeling With Partial Model Exploitation

• URL: http://arxiv.org/abs/2012.12203v1
• Date: Tue, 22 Dec 2020 17:48:32 GMT
• Title: Autonomous sPOMDP Environment Modeling With Partial Model Exploitation
• Authors: Andrew Wilhelm, Aaron Wilhelm, Garrett Fosdick
• Abstract summary: We present a novel state space exploration algorithm by extending the original surprise-based partially-observable Markov Decision Processes (sPOMDP) We show the proposed model significantly increases efficiency and scalability of the original sPOMDP learning techniques with a range of 31-63% gain in training speed. Our results pave the way for extending sPOMDP solutions to a broader set of environments.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: A state space representation of an environment is a classic and yet powerful tool used by many autonomous robotic systems for efficient and often optimal solution planning. However, designing these representations with high performance is laborious and costly, necessitating an effective and versatile tool for autonomous generation of state spaces for autonomous robots. We present a novel state space exploration algorithm by extending the original surprise-based partially-observable Markov Decision Processes (sPOMDP), and demonstrate its effective long-term exploration planning performance in various environments. Through extensive simulation experiments, we show the proposed model significantly increases efficiency and scalability of the original sPOMDP learning techniques with a range of 31-63% gain in training speed while improving robustness in environments with less deterministic transitions. Our results pave the way for extending sPOMDP solutions to a broader set of environments.

Related papers

• Machine Learning-Accelerated Multi-Objective Design of Fractured Geothermal Systems [17.040963667188525]
  We report an Active Learning enhanced Evolutionary Multi-objective Optimization algorithm, integrated with hydrothermal simulations in fractured media. Results demonstrate that the ALEMO approach achieves a remarkable reduction in required simulations, with a speed-up of 1-2 orders of magnitude (10-100 times faster) than traditional evolutionary methods.
  arXiv  Detail & Related papers  (2024-11-01T10:39:23Z)
• R-AIF: Solving Sparse-Reward Robotic Tasks from Pixels with Active Inference and World Models [50.19174067263255]
  We introduce prior preference learning techniques and self-revision schedules to help the agent excel in sparse-reward, continuous action, goal-based robotic control POMDP environments. We show that our agents offer improved performance over state-of-the-art models in terms of cumulative rewards, relative stability, and success rate.
  arXiv  Detail & Related papers  (2024-09-21T18:32:44Z)
• MambaLoc: Efficient Camera Localisation via State Space Model [42.85368902409545]
  Location information is pivotal for the automation and intelligence of terminal devices and edge-cloud IoT systems, such as autonomous vehicles and augmented reality. achieving reliable positioning across diverse IoT applications remains challenging due to significant training costs and the necessity of densely collected data. We have innovatively applied the selective state space (SSM) model to visual localization, introducing a new model named MambaLoc.
  arXiv  Detail & Related papers  (2024-08-19T03:38:29Z)
• A Meta-Engine Framework for Interleaved Task and Motion Planning using Topological Refinements [51.54559117314768]
  Task And Motion Planning (TAMP) is the problem of finding a solution to an automated planning problem. We propose a general and open-source framework for modeling and benchmarking TAMP problems. We introduce an innovative meta-technique to solve TAMP problems involving moving agents and multiple task-state-dependent obstacles.
  arXiv  Detail & Related papers  (2024-08-11T14:57:57Z)
• Machine Learning Insides OptVerse AI Solver: Design Principles and Applications [74.67495900436728]
  We present a comprehensive study on the integration of machine learning (ML) techniques into Huawei Cloud's OptVerse AI solver. We showcase our methods for generating complex SAT and MILP instances utilizing generative models that mirror multifaceted structures of real-world problem. We detail the incorporation of state-of-the-art parameter tuning algorithms which markedly elevate solver performance.
  arXiv  Detail & Related papers  (2024-01-11T15:02:15Z)
• DREAM: Decentralized Reinforcement Learning for Exploration and Efficient Energy Management in Multi-Robot Systems [14.266876062352424]
  Resource-constrained robots often suffer from energy inefficiencies, underutilized computational abilities due to inadequate task allocation, and a lack of robustness in dynamic environments. This paper introduces DREAM - Decentralized Reinforcement Learning for Exploration and Efficient Energy Management in Multi-Robot Systems.
  arXiv  Detail & Related papers  (2023-09-29T17:43:41Z)
• Distributionally Robust Model-based Reinforcement Learning with Large State Spaces [55.14361269378122]
  Three major challenges in reinforcement learning are the complex dynamical systems with large state spaces, the costly data acquisition processes, and the deviation of real-world dynamics from the training environment deployment. We study distributionally robust Markov decision processes with continuous state spaces under the widely used Kullback-Leibler, chi-square, and total variation uncertainty sets. We propose a model-based approach that utilizes Gaussian Processes and the maximum variance reduction algorithm to efficiently learn multi-output nominal transition dynamics.
  arXiv  Detail & Related papers  (2023-09-05T13:42:11Z)
• Obstacle Avoidance for Robotic Manipulator in Joint Space via Improved Proximal Policy Optimization [6.067589886362815]
  In this paper, we train a deep neural network via an improved Proximal Policy Optimization (PPO) algorithm to map from task space to joint space for a 6-DoF manipulator. Since training such a task in real-robot is time-consuming and strenuous, we develop a simulation environment to train the model. Experimental results showed that using our method, the robot was capable of tracking a single target or reaching multiple targets in unstructured environments.
  arXiv  Detail & Related papers  (2022-10-03T10:21:57Z)
• Learning Space Partitions for Path Planning [54.475949279050596]
  PlaLaM outperforms existing path planning methods in 2D navigation tasks, especially in the presence of difficult-to-escape local optima. These gains transfer to highly multimodal real-world tasks, where we outperform strong baselines in compiler phase ordering by up to 245% and in molecular design by up to 0.4 on properties on a 0-1 scale.
  arXiv  Detail & Related papers  (2021-06-19T18:06:11Z)
• Scalable Multi-Robot System for Non-myopic Spatial Sampling [9.37678298330157]
  This paper presents a scalable distributed multi-robot planning algorithm for non-uniform sampling of spatial fields. We analyze the effect of communication between multiple robots, acting independently, on the overall sampling performance of the team.
  arXiv  Detail & Related papers  (2021-05-20T20:30:10Z)
• Localized active learning of Gaussian process state space models [63.97366815968177]
  A globally accurate model is not required to achieve good performance in many common control applications. We propose an active learning strategy for Gaussian process state space models that aims to obtain an accurate model on a bounded subset of the state-action space. By employing model predictive control, the proposed technique integrates information collected during exploration and adaptively improves its exploration strategy.
  arXiv  Detail & Related papers  (2020-05-04T05:35:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.