Multi-Objective Planning with Contextual Lexicographic Reward Preferences

• URL: http://arxiv.org/abs/2502.10476v1
• Date: Thu, 13 Feb 2025 19:04:22 GMT
• Title: Multi-Objective Planning with Contextual Lexicographic Reward Preferences
• Authors: Pulkit Rustagi, Yashwanthi Anand, Sandhya Saisubramanian,
• Abstract summary: We present a framework that enables planning under varying lexicographic objective orderings, depending on context. In a CLMDP, both the objective ordering at a state and the associated reward functions are determined by the context. Our algorithm to solve a CLMDP first computes a policy for each objective ordering and then combines them into a single context-aware policy.
• Score: 5.207917381770368
• License:
• Abstract: Autonomous agents are often required to plan under multiple objectives whose preference ordering varies based on context. The agent may encounter multiple contexts during its course of operation, each imposing a distinct lexicographic ordering over the objectives, with potentially different reward functions associated with each context. Existing approaches to multi-objective planning typically consider a single preference ordering over the objectives, across the state space, and do not support planning under multiple objective orderings within an environment. We present Contextual Lexicographic Markov Decision Process (CLMDP), a framework that enables planning under varying lexicographic objective orderings, depending on the context. In a CLMDP, both the objective ordering at a state and the associated reward functions are determined by the context. We employ a Bayesian approach to infer a state-context mapping from expert trajectories. Our algorithm to solve a CLMDP first computes a policy for each objective ordering and then combines them into a single context-aware policy that is valid and cycle-free. The effectiveness of the proposed approach is evaluated in simulation and using a mobile robot.

Related papers

• Preference-Based Planning in Stochastic Environments: From Partially-Ordered Temporal Goals to Most Preferred Policies [25.731912021122287]
  We consider systems modeled as Markov decision processes, given a partially ordered preference over a set of temporally extended goals. To plan with the partially ordered preference, we introduce order theory to map a preference over temporal goals to a preference over policies for the MDP. A most preferred policy under a ordering induces a nondominated probability distribution over the finite paths in the MDP.
  arXiv  Detail & Related papers  (2024-03-27T02:46:09Z)
• Learning adaptive planning representations with natural language guidance [90.24449752926866]
  This paper describes Ada, a framework for automatically constructing task-specific planning representations. Ada interactively learns a library of planner-compatible high-level action abstractions and low-level controllers adapted to a particular domain of planning tasks.
  arXiv  Detail & Related papers  (2023-12-13T23:35:31Z)
• Probabilistic Planning with Partially Ordered Preferences over Temporal Goals [22.77805882908817]
  We study planning in Markov decision processes (MDPs) with preferences over temporally extended goals. We introduce a variant of deterministic finite automaton, referred to as a preference DFA, for specifying the user's preferences over temporally extended goals. We prove that a weak-stochastic nondominated policy given the preference specification is optimal in the constructed multi-objective MDP.
  arXiv  Detail & Related papers  (2022-09-25T17:13:24Z)
• Planning to Practice: Efficient Online Fine-Tuning by Composing Goals in Latent Space [76.46113138484947]
  General-purpose robots require diverse repertoires of behaviors to complete challenging tasks in real-world unstructured environments. To address this issue, goal-conditioned reinforcement learning aims to acquire policies that can reach goals for a wide range of tasks on command. We propose Planning to Practice, a method that makes it practical to train goal-conditioned policies for long-horizon tasks.
  arXiv  Detail & Related papers  (2022-05-17T06:58:17Z)
• Retrieve-and-Fill for Scenario-based Task-Oriented Semantic Parsing [110.4684789199555]
  We introduce scenario-based semantic parsing: a variant of the original task which first requires disambiguating an utterance's "scenario" This formulation enables us to isolate coarse-grained and fine-grained aspects of the task, each of which we solve with off-the-shelf neural modules. Our model is modular, differentiable, interpretable, and allows us to garner extra supervision from scenarios.
  arXiv  Detail & Related papers  (2022-02-02T08:00:21Z)
• Learning Models as Functionals of Signed-Distance Fields for Manipulation Planning [51.74463056899926]
  This work proposes an optimization-based manipulation planning framework where the objectives are learned functionals of signed-distance fields that represent objects in the scene. We show that representing objects as signed-distance fields not only enables to learn and represent a variety of models with higher accuracy compared to point-cloud and occupancy measure representations.
  arXiv  Detail & Related papers  (2021-10-02T12:36:58Z)
• Exploring Relational Context for Multi-Task Dense Prediction [76.86090370115]
  We consider a multi-task environment for dense prediction tasks, represented by a common backbone and independent task-specific heads. We explore various attention-based contexts, such as global and local, in the multi-task setting. We propose an Adaptive Task-Relational Context module, which samples the pool of all available contexts for each task pair.
  arXiv  Detail & Related papers  (2021-04-28T16:45:56Z)
• A Distributional View on Multi-Objective Policy Optimization [24.690800846837273]
  We propose an algorithm for multi-objective reinforcement learning that enables setting desired preferences for objectives in a scale-invariant way. We show that setting different preferences in our framework allows us to trace out the space of nondominated solutions.
  arXiv  Detail & Related papers  (2020-05-15T13:02:17Z)
• Parallel processor scheduling: formulation as multi-objective linguistic optimization and solution using Perceptual Reasoning based methodology [13.548237279353408]
  The aim of the scheduling policy is to achieve the optimal value of an objective, like production time, cost, etc. The experts generally provide their opinions, about various scheduling criteria (pertaining to the scheduling policies) in linguistic terms or words. We have also compared the results of the PR based solution methodology with those obtained from the 2-tuple based solution methodology.
  arXiv  Detail & Related papers  (2020-04-30T17:04:49Z)
• Divide-and-Conquer Monte Carlo Tree Search For Goal-Directed Planning [78.65083326918351]
  We consider alternatives to an implicit sequential planning assumption. We propose Divide-and-Conquer Monte Carlo Tree Search (DC-MCTS) for approximating the optimal plan. We show that this algorithmic flexibility over planning order leads to improved results in navigation tasks in grid-worlds.
  arXiv  Detail & Related papers  (2020-04-23T18:08:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.