Autocorrelation effects in a stochastic-process model for decision making via time series

• URL: http://arxiv.org/abs/2603.05559v1
• Date: Thu, 05 Mar 2026 09:39:09 GMT
• Title: Autocorrelation effects in a stochastic-process model for decision making via time series
• Abstract summary: Decision makers exploit photonic chaotic dynamics obtained by semiconductor lasers to solve multi-armed bandit problems.<n>Experiments have reported that decision accuracy strongly on this autocorrelation property.<n>This study paves the way for improving the decision-making scheme for reinforcement learning applications in wireless communications and robotics.
• Score: 0.28707625120094377
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Decision makers exploiting photonic chaotic dynamics obtained by semiconductor lasers provide an ultrafast approach to solving multi-armed bandit problems by using a temporal optical signal as the driving source for sequential decisions. In such systems, the sampling interval of the chaotic waveform shapes the temporal correlation of the resulting time series, and experiments have reported that decision accuracy depends strongly on this autocorrelation property. However, it remains unclear whether the benefit of autocorrelation can be explained by a minimal mathematical model. Here, we analyze a stochastic-process model of the time-series-based decision making using the tug-of-war principle for solving the two-armed bandit problem, where the threshold and a two-valued Markov signal evolve jointly. Numerical results reveal an environment-dependent structure: negative (positive) autocorrelation is optimal in reward-rich (reward-poor) environments. These findings show that negative autocorrelation of the time series is advantageous when the sum of the winning probabilities is more than $1$, whereas positive autocorrelation is useful when the sum of the winning probabilities is less than $1$. Moreover, the performance is independent of autocorrelation if the sum of the winning probabilities equals $1$, which is mathematically clarified. This study paves the way for improving the decision-making scheme for reinforcement learning applications in wireless communications and robotics.

Related papers

• Dual-domain fused LSTM modeling for efficient time-dependent reliability analysis [25.846824628836917]
  Time-dependent reliability analysis is crucial for ensuring the long-term safety and performance of engineering systems under uncertainties.<n>This paper proposes a dual-domain fused long short-term memory (DDF-LSTM) model for efficient and accurate time-dependent reliability analysis.
  arXiv  Detail & Related papers  (2026-06-30T15:39:47Z)
• Precise: SDE-Consistent Stochastic Sampling for RL Post-Training of Flow-Matching Models [56.67321805551389]
  Reinforcement learning (RL) has become an effective way to improve prompt alignment and perceptual quality in diffusion and flow-matching generators.<n>The sampler, controlling the exploration behavior and denoising dynamics, is thus part of the policy.<n>We propose a new sampler that balances effective exploration with stability.
  arXiv  Detail & Related papers  (2026-05-22T11:37:22Z)
• Harmonic Path Integral Diffusion [0.4527270266697462]
  We present a novel approach for sampling from a continuous multivariate probability distribution, which may either be explicitly known (up to a normalization factor) or represented via empirical samples. Our method constructs a time-dependent bridge from a delta function centered at the origin of the state space at $t=0$, transforming it into the target distribution at $t=1$. We contrast these algorithms with other sampling methods, particularly simulated and path integral sampling, highlighting their advantages in terms of analytical control, accuracy, and computational efficiency.
  arXiv  Detail & Related papers  (2024-09-23T16:20:21Z)
• Exact Recovery Guarantees for Parameterized Nonlinear System Identification Problem under Sparse Disturbances or Semi-Oblivious Attacks [16.705631360131886]
  We study the problem of learning a nonlinear dynamical system by parameterizing its dynamics using basis functions.<n>We show that finite-time exact recovery is achieved with high probability, even when $p$ approaches 1.
  arXiv  Detail & Related papers  (2024-08-30T22:12:57Z)
• Causal Feature Selection via Transfer Entropy [59.999594949050596]
  Causal discovery aims to identify causal relationships between features with observational data. We introduce a new causal feature selection approach that relies on the forward and backward feature selection procedures. We provide theoretical guarantees on the regression and classification errors for both the exact and the finite-sample cases.
  arXiv  Detail & Related papers  (2023-10-17T08:04:45Z)
• PerturbScore: Connecting Discrete and Continuous Perturbations in NLP [64.28423650146877]
  In this paper, we aim to connect discrete perturbations with continuous perturbations. We design a regression task as a PerturbScore to learn the correlation automatically. We find that we can build a connection between discrete and continuous perturbations and use the proposed PerturbScore to learn such correlation.
  arXiv  Detail & Related papers  (2023-10-13T06:50:15Z)
• Seeing is not Believing: Robust Reinforcement Learning against Spurious Correlation [57.351098530477124]
  We consider one critical type of robustness against spurious correlation, where different portions of the state do not have correlations induced by unobserved confounders. A model that learns such useless or even harmful correlation could catastrophically fail when the confounder in the test case deviates from the training one. Existing robust algorithms that assume simple and unstructured uncertainty sets are therefore inadequate to address this challenge.
  arXiv  Detail & Related papers  (2023-07-15T23:53:37Z)
• Continuous-Time Modeling of Counterfactual Outcomes Using Neural Controlled Differential Equations [84.42837346400151]
  Estimating counterfactual outcomes over time has the potential to unlock personalized healthcare. Existing causal inference approaches consider regular, discrete-time intervals between observations and treatment decisions. We propose a controllable simulation environment based on a model of tumor growth for a range of scenarios.
  arXiv  Detail & Related papers  (2022-06-16T17:15:15Z)
• Theory of Acceleration of Decision Making by Correlated Times Sequences [0.15658704610960572]
  Photonic accelerators have been intensively studied to provide enhanced information processing capability. Laser chaos provides the ability to solve multi-armed bandit (MAB) problems or decision-making problems at GHz order. The underlying mechanism of why decision-making is accelerated by correlated time sequence is unknown.
  arXiv  Detail & Related papers  (2022-03-30T02:27:01Z)
• Stochastic Optimization under Distributional Drift [3.0229888038442922]
  We provide non-asymptotic convergence guarantees for algorithms with iterate averaging, focusing on bounds valid both in expectation and with high probability. We identify a low drift-to-noise regime in which the tracking efficiency of the gradient method benefits significantly from a step decay schedule.
  arXiv  Detail & Related papers  (2021-08-16T21:57:39Z)
• Momentum Accelerates the Convergence of Stochastic AUPRC Maximization [80.8226518642952]
  We study optimization of areas under precision-recall curves (AUPRC), which is widely used for imbalanced tasks. We develop novel momentum methods with a better iteration of $O (1/epsilon4)$ for finding an $epsilon$stationary solution. We also design a novel family of adaptive methods with the same complexity of $O (1/epsilon4)$, which enjoy faster convergence in practice.
  arXiv  Detail & Related papers  (2021-07-02T16:21:52Z)
• Navigating to the Best Policy in Markov Decision Processes [68.8204255655161]
  We investigate the active pure exploration problem in Markov Decision Processes. Agent sequentially selects actions and, from the resulting system trajectory, aims at the best as fast as possible.
  arXiv  Detail & Related papers  (2021-06-05T09:16:28Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.