Guided Exploration for Efficient Relational Model Learning

• URL: http://arxiv.org/abs/2502.06146v2
• Date: Sat, 10 May 2025 04:07:03 GMT
• Title: Guided Exploration for Efficient Relational Model Learning
• Authors: Annie Feng, Nishanth Kumar, Tomas Lozano-Perez, Leslie Pack-Kaelbling,
• Abstract summary: Efficient exploration is critical for learning relational models in large-scale environments with complex, long-horizon tasks.<n>Goal-literal babbling (GLIB) improves upon random exploration by setting and planning to novel goals.<n>We introduce Baking-Large, a challenging domain with extensive state-action spaces and long-horizon tasks.
• Score: 20.853275384388805
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Efficient exploration is critical for learning relational models in large-scale environments with complex, long-horizon tasks. Random exploration methods often collect redundant or irrelevant data, limiting their ability to learn accurate relational models of the environment. Goal-literal babbling (GLIB) improves upon random exploration by setting and planning to novel goals, but its reliance on random actions and random novel goal selection limits its scalability to larger domains. In this work, we identify the principles underlying efficient exploration in relational domains: (1) operator initialization with demonstrations that cover the distinct lifted effects necessary for planning and (2) refining preconditions to collect maximally informative transitions by selecting informative goal-action pairs and executing plans to them. To demonstrate these principles, we introduce Baking-Large, a challenging domain with extensive state-action spaces and long-horizon tasks. We evaluate methods using oracle-driven demonstrations for operator initialization and precondition-targeting guidance to efficiently gather critical transitions. Experiments show that both the oracle demonstrations and precondition-targeting oracle guidance significantly improve sample efficiency and generalization, paving the way for future methods to use these principles to efficiently learn accurate relational models in complex domains.

Related papers

• Beyond Policy Optimization: A Data Curation Flywheel for Sparse-Reward Long-Horizon Planning [15.103861901247125]
  We propose a three-stage framework to develop robust reasoning models for sparse environments.<n>Our framework bootstraps efficient reasoning using the proposed planning quaternions with long-short chain-of-thought fusion.<n>Experiments on ALFWorld, ScienceWorld, and WebShop demonstrate that our approach achieves state-of-the-art with significant token efficiency.
  arXiv  Detail & Related papers  (2025-08-05T02:56:58Z)
• Learning to Explore: An In-Context Learning Approach for Pure Exploration [23.16863295063427]
  In this work, we study the active sequential hypothesis testing problem, also known as pure exploration.<n>We introduce In-Context Pure Exploration (ICPE), an in-context learning approach that uses Transformers to learn exploration strategies directly from experience.<n>ICPE combines supervised learning and reinforcement learning to identify and exploit latent structure across related tasks, without requiring prior assumptions.
  arXiv  Detail & Related papers  (2025-06-02T17:04:50Z)
• Rethinking Multi-Objective Learning through Goal-Conditioned Supervised Learning [8.593384839118658]
  Multi-objective learning aims to optimize multiple objectives simultaneously with a single model.<n>It suffers from the difficulty to formalize and conduct the exact learning process.<n>We propose a general framework for automatically learning to achieve multiple objectives based on the existing sequential data.
  arXiv  Detail & Related papers  (2024-12-12T03:47:40Z)
• Learn from the Learnt: Source-Free Active Domain Adaptation via Contrastive Sampling and Visual Persistence [60.37934652213881]
  Domain Adaptation (DA) facilitates knowledge transfer from a source domain to a related target domain. This paper investigates a practical DA paradigm, namely Source data-Free Active Domain Adaptation (SFADA), where source data becomes inaccessible during adaptation. We present learn from the learnt (LFTL), a novel paradigm for SFADA to leverage the learnt knowledge from the source pretrained model and actively iterated models without extra overhead.
  arXiv  Detail & Related papers  (2024-07-26T17:51:58Z)
• Parameter-Efficient Active Learning for Foundational models [7.799711162530711]
  Foundational vision transformer models have shown impressive few shot performance on many vision tasks. This research presents a novel investigation into the application of parameter efficient fine-tuning methods within an active learning (AL) framework.
  arXiv  Detail & Related papers  (2024-06-13T16:30:32Z)
• Entropy-Regularized Token-Level Policy Optimization for Language Agent Reinforcement [67.1393112206885]
  Large Language Models (LLMs) have shown promise as intelligent agents in interactive decision-making tasks. We introduce Entropy-Regularized Token-level Policy Optimization (ETPO), an entropy-augmented RL method tailored for optimizing LLMs at the token level. We assess the effectiveness of ETPO within a simulated environment that models data science code generation as a series of multi-step interactive tasks.
  arXiv  Detail & Related papers  (2024-02-09T07:45:26Z)
• Towards Robust Continual Learning with Bayesian Adaptive Moment Regularization [51.34904967046097]
  Continual learning seeks to overcome the challenge of catastrophic forgetting, where a model forgets previously learnt information. We introduce a novel prior-based method that better constrains parameter growth, reducing catastrophic forgetting. Results show that BAdam achieves state-of-the-art performance for prior-based methods on challenging single-headed class-incremental experiments.
  arXiv  Detail & Related papers  (2023-09-15T17:10:51Z)
• An unsupervised deep learning framework via integrated optimization of representation learning and GMM-based modeling [31.334196673143257]
  This paper introduces a new principle of joint learning on both deep representations and GMM-based deep modeling. In comparison with the existing work in similar areas, our objective function has two learning targets, which are created to be jointly optimized. The compactness of clusters is significantly enhanced by reducing the intra-cluster distances, and the separability is improved by increasing the inter-cluster distances.
  arXiv  Detail & Related papers  (2020-09-11T04:57:03Z)
• Sequential Transfer in Reinforcement Learning with a Generative Model [48.40219742217783]
  We show how to reduce the sample complexity for learning new tasks by transferring knowledge from previously-solved ones. We derive PAC bounds on its sample complexity which clearly demonstrate the benefits of using this kind of prior knowledge. We empirically verify our theoretical findings in simple simulated domains.
  arXiv  Detail & Related papers  (2020-07-01T19:53:35Z)
• A Trainable Optimal Transport Embedding for Feature Aggregation and its Relationship to Attention [96.77554122595578]
  We introduce a parametrized representation of fixed size, which embeds and then aggregates elements from a given input set according to the optimal transport plan between the set and a trainable reference. Our approach scales to large datasets and allows end-to-end training of the reference, while also providing a simple unsupervised learning mechanism with small computational cost.
  arXiv  Detail & Related papers  (2020-06-22T08:35:58Z)
• Goal-Directed Planning for Habituated Agents by Active Inference Using a Variational Recurrent Neural Network [5.000272778136268]
  This study shows that the predictive coding (PC) and active inference (AIF) frameworks can develop better generalization by learning a prior distribution in a low dimensional latent state space. In our proposed model, learning is carried out by inferring optimal latent variables as well as synaptic weights for maximizing the evidence lower bound. Our proposed model was evaluated with both simple and complex robotic tasks in simulation, which demonstrated sufficient generalization in learning with limited training data.
  arXiv  Detail & Related papers  (2020-05-27T06:43:59Z)
• Localized active learning of Gaussian process state space models [63.97366815968177]
  A globally accurate model is not required to achieve good performance in many common control applications. We propose an active learning strategy for Gaussian process state space models that aims to obtain an accurate model on a bounded subset of the state-action space. By employing model predictive control, the proposed technique integrates information collected during exploration and adaptively improves its exploration strategy.
  arXiv  Detail & Related papers  (2020-05-04T05:35:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.