Modelling bounded rational decision-making through Wasserstein constraints

• URL: http://arxiv.org/abs/2504.03743v1
• Date: Tue, 01 Apr 2025 15:19:34 GMT
• Title: Modelling bounded rational decision-making through Wasserstein constraints
• Authors: Benjamin Patrick Evans, Leo Ardon, Sumitra Ganesh,
• Abstract summary: Modelling bounded rational decision-making through information constrained processing provides a principled approach.<n>Existing approaches are generally based on Entropy, Kullback-Leibler divergence, or Mutual Information.<n>We propose an alternative approach for modeling bounded rational RL agents utilising Wasserstein distances.
• Score: 3.3161769688599025
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Modelling bounded rational decision-making through information constrained processing provides a principled approach for representing departures from rationality within a reinforcement learning framework, while still treating decision-making as an optimization process. However, existing approaches are generally based on Entropy, Kullback-Leibler divergence, or Mutual Information. In this work, we highlight issues with these approaches when dealing with ordinal action spaces. Specifically, entropy assumes uniform prior beliefs, missing the impact of a priori biases on decision-makings. KL-Divergence addresses this, however, has no notion of "nearness" of actions, and additionally, has several well known potentially undesirable properties such as the lack of symmetry, and furthermore, requires the distributions to have the same support (e.g. positive probability for all actions). Mutual information is often difficult to estimate. Here, we propose an alternative approach for modeling bounded rational RL agents utilising Wasserstein distances. This approach overcomes the aforementioned issues. Crucially, this approach accounts for the nearness of ordinal actions, modeling "stickiness" in agent decisions and unlikeliness of rapidly switching to far away actions, while also supporting low probability actions, zero-support prior distributions, and is simple to calculate directly.

Related papers

• Unifying Perplexing Behaviors in Modified BP Attributions through Alignment Perspective [61.5509267439999]
  We present a unified theoretical framework for methods like GBP, RectGrad, LRP, and DTD.<n>We demonstrate that they achieve input alignment by combining the weights of activated neurons.<n>This alignment improves the visualization quality and reduces sensitivity to weight randomization.
  arXiv  Detail & Related papers  (2025-03-14T07:58:26Z)
• Robust Counterfactual Inference in Markov Decision Processes [1.5197843979051473]
  Current approaches assume a specific causal model to make counterfactuals identifiable.<n>We propose a novel non-parametric approach that computes tight bounds on counterfactual transition probabilities.
  arXiv  Detail & Related papers  (2025-02-19T13:56:20Z)
• Provably Efficient UCB-type Algorithms For Learning Predictive State Representations [55.00359893021461]
  The sequential decision-making problem is statistically learnable if it admits a low-rank structure modeled by predictive state representations (PSRs) This paper proposes the first known UCB-type approach for PSRs, featuring a novel bonus term that upper bounds the total variation distance between the estimated and true models. In contrast to existing approaches for PSRs, our UCB-type algorithms enjoy computational tractability, last-iterate guaranteed near-optimal policy, and guaranteed model accuracy.
  arXiv  Detail & Related papers  (2023-07-01T18:35:21Z)
• Composed Image Retrieval with Text Feedback via Multi-grained Uncertainty Regularization [73.04187954213471]
  We introduce a unified learning approach to simultaneously modeling the coarse- and fine-grained retrieval. The proposed method has achieved +4.03%, +3.38%, and +2.40% Recall@50 accuracy over a strong baseline.
  arXiv  Detail & Related papers  (2022-11-14T14:25:40Z)
• Data-Driven Influence Functions for Optimization-Based Causal Inference [105.5385525290466]
  We study a constructive algorithm that approximates Gateaux derivatives for statistical functionals by finite differencing. We study the case where probability distributions are not known a priori but need to be estimated from data.
  arXiv  Detail & Related papers  (2022-08-29T16:16:22Z)
• Entropy, Information, and the Updating of Probabilities [0.0]
  This paper is a review of a particular approach to the method of maximum entropy as a general framework for inference. The ME method goes beyond the mere selection of a single posterior, but also addresses the question of how much less probable other distributions might be.
  arXiv  Detail & Related papers  (2021-07-09T16:27:23Z)
• Algorithmic Recourse in Partially and Fully Confounded Settings Through Bounding Counterfactual Effects [0.6299766708197883]
  Algorithmic recourse aims to provide actionable recommendations to individuals to obtain a more favourable outcome from an automated decision-making system. Existing methods compute the effect of recourse actions using a causal model learnt from data under the assumption of no hidden confounding and modelling assumptions such as additive noise. We propose an alternative approach for discrete random variables which relaxes these assumptions and allows for unobserved confounding and arbitrary structural equations.
  arXiv  Detail & Related papers  (2021-06-22T15:07:49Z)
• Identification of Unexpected Decisions in Partially Observable Monte-Carlo Planning: a Rule-Based Approach [78.05638156687343]
  We propose a methodology for analyzing POMCP policies by inspecting their traces. The proposed method explores local properties of policy behavior to identify unexpected decisions. We evaluate our approach on Tiger, a standard benchmark for POMDPs, and a real-world problem related to mobile robot navigation.
  arXiv  Detail & Related papers  (2020-12-23T15:09:28Z)
• Stein Variational Model Predictive Control [130.60527864489168]
  Decision making under uncertainty is critical to real-world, autonomous systems. Model Predictive Control (MPC) methods have demonstrated favorable performance in practice, but remain limited when dealing with complex distributions. We show that this framework leads to successful planning in challenging, non optimal control problems.
  arXiv  Detail & Related papers  (2020-11-15T22:36:59Z)
• Algorithmic recourse under imperfect causal knowledge: a probabilistic approach [15.124107808802703]
  We show that it is impossible to guarantee recourse without access to the true structural equations. We propose two probabilistic approaches to select optimal actions that achieve recourse with high probability given limited causal knowledge.
  arXiv  Detail & Related papers  (2020-06-11T21:19:07Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.