HMM for Discovering Decision-Making Dynamics Using Reinforcement Learning Experiments

• URL: http://arxiv.org/abs/2401.13929v2
• Date: Fri, 26 Jul 2024 01:12:39 GMT
• Title: HMM for Discovering Decision-Making Dynamics Using Reinforcement Learning Experiments
• Authors: Xingche Guo, Donglin Zeng, Yuanjia Wang,
• Abstract summary: Evidence indicates that reward processing abnormalities may serve as a behavioral marker for MDD. Recent findings suggest the inadequacy of characterizing reward learning solely based on a single RL model. We propose a novel RL-HMM framework for analyzing reward-based decision-making.
• Score: 5.857093069873734
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Major depressive disorder (MDD) presents challenges in diagnosis and treatment due to its complex and heterogeneous nature. Emerging evidence indicates that reward processing abnormalities may serve as a behavioral marker for MDD. To measure reward processing, patients perform computer-based behavioral tasks that involve making choices or responding to stimulants that are associated with different outcomes. Reinforcement learning (RL) models are fitted to extract parameters that measure various aspects of reward processing to characterize how patients make decisions in behavioral tasks. Recent findings suggest the inadequacy of characterizing reward learning solely based on a single RL model; instead, there may be a switching of decision-making processes between multiple strategies. An important scientific question is how the dynamics of learning strategies in decision-making affect the reward learning ability of individuals with MDD. Motivated by the probabilistic reward task (PRT) within the EMBARC study, we propose a novel RL-HMM framework for analyzing reward-based decision-making. Our model accommodates learning strategy switching between two distinct approaches under a hidden Markov model (HMM): subjects making decisions based on the RL model or opting for random choices. We account for continuous RL state space and allow time-varying transition probabilities in the HMM. We introduce a computationally efficient EM algorithm for parameter estimation and employ a nonparametric bootstrap for inference. We apply our approach to the EMBARC study to show that MDD patients are less engaged in RL compared to the healthy controls, and engagement is associated with brain activities in the negative affect circuitry during an emotional conflict task.

Related papers

• Querying Easily Flip-flopped Samples for Deep Active Learning [63.62397322172216]
  Active learning is a machine learning paradigm that aims to improve the performance of a model by strategically selecting and querying unlabeled data. One effective selection strategy is to base it on the model's predictive uncertainty, which can be interpreted as a measure of how informative a sample is. This paper proposes the it least disagree metric (LDM) as the smallest probability of disagreement of the predicted label.
  arXiv  Detail & Related papers  (2024-01-18T08:12:23Z)
• Cross-modality Guidance-aided Multi-modal Learning with Dual Attention for MRI Brain Tumor Grading [47.50733518140625]
  Brain tumor represents one of the most fatal cancers around the world, and is very common in children and the elderly. We propose a novel cross-modality guidance-aided multi-modal learning with dual attention for addressing the task of MRI brain tumor grading.
  arXiv  Detail & Related papers  (2024-01-17T07:54:49Z)
• GEC: A Unified Framework for Interactive Decision Making in MDP, POMDP, and Beyond [101.5329678997916]
  We study sample efficient reinforcement learning (RL) under the general framework of interactive decision making. We propose a novel complexity measure, generalized eluder coefficient (GEC), which characterizes the fundamental tradeoff between exploration and exploitation. We show that RL problems with low GEC form a remarkably rich class, which subsumes low Bellman eluder dimension problems, bilinear class, low witness rank problems, PO-bilinear class, and generalized regular PSR.
  arXiv  Detail & Related papers  (2022-11-03T16:42:40Z)
• Offline Reinforcement Learning with Instrumental Variables in Confounded Markov Decision Processes [93.61202366677526]
  We study the offline reinforcement learning (RL) in the face of unmeasured confounders. We propose various policy learning methods with the finite-sample suboptimality guarantee of finding the optimal in-class policy.
  arXiv  Detail & Related papers  (2022-09-18T22:03:55Z)
• MRI-based Multi-task Decoupling Learning for Alzheimer's Disease Detection and MMSE Score Prediction: A Multi-site Validation [9.427540028148963]
  Accurately detecting Alzheimer's disease (AD) and predicting mini-mental state examination (MMSE) score are important tasks in elderly health by magnetic resonance imaging (MRI) Most of the previous methods on these two tasks are based on single-task learning and rarely consider the correlation between them. We propose a MRI-based multi-task decoupled learning method for AD detection and MMSE score prediction.
  arXiv  Detail & Related papers  (2022-04-02T09:19:18Z)
• Ambiguous Dynamic Treatment Regimes: A Reinforcement Learning Approach [0.0]
  Dynamic Treatment Regimes (DTRs) are widely studied to formalize this process. We develop Reinforcement Learning methods to efficiently learn optimal treatment regimes.
  arXiv  Detail & Related papers  (2021-12-08T20:22:04Z)
• Identification of brain states, transitions, and communities using functional MRI [0.5872014229110214]
  We propose a Bayesian model-based characterization of latent brain states and showcase a novel method based on posterior predictive discrepancy. Our results obtained through an analysis of task-fMRI data show appropriate lags between external task demands and change-points between brain states.
  arXiv  Detail & Related papers  (2021-01-26T08:10:00Z)
• Adversarial Sample Enhanced Domain Adaptation: A Case Study on Predictive Modeling with Electronic Health Records [57.75125067744978]
  We propose a data augmentation method to facilitate domain adaptation. adversarially generated samples are used during domain adaptation. Results confirm the effectiveness of our method and the generality on different tasks.
  arXiv  Detail & Related papers  (2021-01-13T03:20:20Z)
• On the Reliability and Generalizability of Brain-inspired Reinforcement Learning Algorithms [10.09712608508383]
  We show that the computational model combining model-based and model-free control, which we term the prefrontal RL, reliably encodes the information of high-level policy that humans learned. This is the first attempt to formally test the possibility that computational models mimicking the way the brain solves general problems can lead to practical solutions.
  arXiv  Detail & Related papers  (2020-07-09T06:32:42Z)
• Unified Models of Human Behavioral Agents in Bandits, Contextual Bandits and RL [28.38826379640553]
  We propose a more general and flexible parametric framework for sequential decision making. Inspired by the known reward processing abnormalities of many mental disorders, our clinically-inspired agents demonstrated interesting behavioral trajectories.
  arXiv  Detail & Related papers  (2020-05-10T01:43:39Z)
• Task-Feature Collaborative Learning with Application to Personalized Attribute Prediction [166.87111665908333]
  We propose a novel multi-task learning method called Task-Feature Collaborative Learning (TFCL) Specifically, we first propose a base model with a heterogeneous block-diagonal structure regularizer to leverage the collaborative grouping of features and tasks. As a practical extension, we extend the base model by allowing overlapping features and differentiating the hard tasks.
  arXiv  Detail & Related papers  (2020-04-29T02:32:04Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.