Quantum-Inspired Reinforcement Learning in the Presence of Epistemic Ambivalence

• URL: http://arxiv.org/abs/2503.04219v1
• Date: Thu, 06 Mar 2025 08:54:31 GMT
• Title: Quantum-Inspired Reinforcement Learning in the Presence of Epistemic Ambivalence
• Authors: Alireza Habibi, Saeed Ghoorchian, Setareh Maghsudi,
• Abstract summary: Epistem ambivalence (EA) emerges from conflicting pieces of evidence or contradictory experiences.<n>EA-MDP aims to understand and control EA in decision-making processes.
• Score: 4.683612295430956
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The complexity of online decision-making under uncertainty stems from the requirement of finding a balance between exploiting known strategies and exploring new possibilities. Naturally, the uncertainty type plays a crucial role in developing decision-making strategies that manage complexity effectively. In this paper, we focus on a specific form of uncertainty known as epistemic ambivalence (EA), which emerges from conflicting pieces of evidence or contradictory experiences. It creates a delicate interplay between uncertainty and confidence, distinguishing it from epistemic uncertainty that typically diminishes with new information. Indeed, ambivalence can persist even after additional knowledge is acquired. To address this phenomenon, we propose a novel framework, called the epistemically ambivalent Markov decision process (EA-MDP), aiming to understand and control EA in decision-making processes. This framework incorporates the concept of a quantum state from the quantum mechanics formalism, and its core is to assess the probability and reward of every possible outcome. We calculate the reward function using quantum measurement techniques and prove the existence of an optimal policy and an optimal value function in the EA-MDP framework. We also propose the EA-epsilon-greedy Q-learning algorithm. To evaluate the impact of EA on decision-making and the expedience of our framework, we study two distinct experimental setups, namely the two-state problem and the lattice problem. Our results show that using our methods, the agent converges to the optimal policy in the presence of EA.

Related papers

• Generalized Decision Focused Learning under Imprecise Uncertainty--Theoretical Study [6.137404366514538]
  Decision Focused Learning has emerged as a critical paradigm for integrating machine learning with downstream optimisation.<n>Existing methodologies predominantly rely on probabilistic models and focus narrowly on task objectives.<n>This paper bridges these gaps by introducing innovative frameworks.
  arXiv  Detail & Related papers  (2025-02-25T08:53:02Z)
• Decision Making in Changing Environments: Robustness, Query-Based Learning, and Differential Privacy [59.64384863882473]
  We study the problem of interactive decision making in which the underlying environment changes over time subject to given constraints.<n>We propose a framework, which provides an complexity between the complexity and adversarial settings of decision making.
  arXiv  Detail & Related papers  (2025-01-24T21:31:50Z)
• Know Where You're Uncertain When Planning with Multimodal Foundation Models: A Formal Framework [54.40508478482667]
  We present a comprehensive framework to disentangle, quantify, and mitigate uncertainty in perception and plan generation. We propose methods tailored to the unique properties of perception and decision-making. We show that our uncertainty disentanglement framework reduces variability by up to 40% and enhances task success rates by 5% compared to baselines.
  arXiv  Detail & Related papers  (2024-11-03T17:32:00Z)
• Uncertainty Quantification for Forward and Inverse Problems of PDEs via Latent Global Evolution [110.99891169486366]
  We propose a method that integrates efficient and precise uncertainty quantification into a deep learning-based surrogate model. Our method endows deep learning-based surrogate models with robust and efficient uncertainty quantification capabilities for both forward and inverse problems. Our method excels at propagating uncertainty over extended auto-regressive rollouts, making it suitable for scenarios involving long-term predictions.
  arXiv  Detail & Related papers  (2024-02-13T11:22:59Z)
• Explaining by Imitating: Understanding Decisions by Interpretable Policy Learning [72.80902932543474]
  Understanding human behavior from observed data is critical for transparency and accountability in decision-making. Consider real-world settings such as healthcare, in which modeling a decision-maker's policy is challenging. We propose a data-driven representation of decision-making behavior that inheres transparency by design, accommodates partial observability, and operates completely offline.
  arXiv  Detail & Related papers  (2023-10-28T13:06:14Z)
• On solving decision and risk management problems subject to uncertainty [91.3755431537592]
  Uncertainty is a pervasive challenge in decision and risk management. This paper develops a systematic understanding of such strategies, determine their range of application, and develop a framework to better employ them.
  arXiv  Detail & Related papers  (2023-01-18T19:16:23Z)
• On the Complexity of Adversarial Decision Making [101.14158787665252]
  We show that the Decision-Estimation Coefficient is necessary and sufficient to obtain low regret for adversarial decision making. We provide new structural results that connect the Decision-Estimation Coefficient to variants of other well-known complexity measures.
  arXiv  Detail & Related papers  (2022-06-27T06:20:37Z)
• Uncertainty quantification and exploration-exploitation trade-off in humans [0.0]
  The main objective of this paper is to outline a theoretical framework to analyse how humans' decision-making strategies under uncertainty manage the trade-off between information gathering (exploration) and reward seeking (exploitation)
  arXiv  Detail & Related papers  (2021-02-05T16:03:04Z)
• Quantum Cognitively Motivated Decision Fusion for Video Sentiment Analysis [22.701975963984378]
  We show that the sentiment judgment from one modality could be incompatible with the judgment from another. We propose a fundamentally new, quantum cognitively motivated fusion strategy for predicting sentiment judgments.
  arXiv  Detail & Related papers  (2021-01-12T11:06:04Z)
• QuLBIT: Quantum-Like Bayesian Inference Technologies for Cognition and Decision [0.11470070927586014]
  This paper provides the foundations of a unified cognitive decision-making framework (QulBIT) which is derived from quantum theory. We detail the main modules of the unified framework, the explanatory analysis method, and illustrate their application in situations violating the Sure Thing Principle.
  arXiv  Detail & Related papers  (2020-05-30T09:02:03Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.