Testing for the Markov Property in Time Series via Deep Conditional Generative Learning

• URL: http://arxiv.org/abs/2305.19244v1
• Date: Tue, 30 May 2023 17:32:00 GMT
• Title: Testing for the Markov Property in Time Series via Deep Conditional Generative Learning
• Authors: Yunzhe Zhou and Chengchun Shi and Lexin Li and Qiwei Yao
• Abstract summary: We propose a nonparametric test for the Markov property in high-dimensional time series via deep conditional generative learning. We show that the test controls the type-I errorally, and has the power approaching one. We derive a doubly robust test statistic, which employs a nonparametric estimation but achieves a parametric convergence rate.
• Score: 6.7826352751791985
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The Markov property is widely imposed in analysis of time series data. Correspondingly, testing the Markov property, and relatedly, inferring the order of a Markov model, are of paramount importance. In this article, we propose a nonparametric test for the Markov property in high-dimensional time series via deep conditional generative learning. We also apply the test sequentially to determine the order of the Markov model. We show that the test controls the type-I error asymptotically, and has the power approaching one. Our proposal makes novel contributions in several ways. We utilize and extend state-of-the-art deep generative learning to estimate the conditional density functions, and establish a sharp upper bound on the approximation error of the estimators. We derive a doubly robust test statistic, which employs a nonparametric estimation but achieves a parametric convergence rate. We further adopt sample splitting and cross-fitting to minimize the conditions required to ensure the consistency of the test. We demonstrate the efficacy of the test through both simulations and the three data applications.

Related papers

• Asymptotically Optimal Sequential Testing with Markovian Data [20.18233038746029]
  We study one-sided and $$-correct sequential hypothesis testing for data generated by an ergodic Markov chain.<n>We establish a tight non-asymptotic-dependent instance on the expected stopping time of any valid sequential test under the alternative.<n>Our findings yield a sharp and general characterization of optimal sequential testing procedures under Markovian dependence.
  arXiv  Detail & Related papers  (2026-02-19T18:11:02Z)
• Statistical Inference for Temporal Difference Learning with Linear Function Approximation [55.80276145563105]
  We investigate the statistical properties of Temporal Difference learning with Polyak-Ruppert averaging.<n>We make three theoretical contributions that improve upon the current state-of-the-art results.
  arXiv  Detail & Related papers  (2024-10-21T15:34:44Z)
• A Kernel-Based Conditional Two-Sample Test Using Nearest Neighbors (with Applications to Calibration, Regression Curves, and Simulation-Based Inference) [3.622435665395788]
  We introduce a kernel-based measure for detecting differences between two conditional distributions. When the two conditional distributions are the same, the estimate has a Gaussian limit and its variance has a simple form that can be easily estimated from the data. We also provide a resampling based test using our estimate that applies to the conditional goodness-of-fit problem.
  arXiv  Detail & Related papers  (2024-07-23T15:04:38Z)
• Uncertainty-Calibrated Test-Time Model Adaptation without Forgetting [55.17761802332469]
  Test-time adaptation (TTA) seeks to tackle potential distribution shifts between training and test data by adapting a given model w.r.t. any test sample. Prior methods perform backpropagation for each test sample, resulting in unbearable optimization costs to many applications. We propose an Efficient Anti-Forgetting Test-Time Adaptation (EATA) method which develops an active sample selection criterion to identify reliable and non-redundant samples.
  arXiv  Detail & Related papers  (2024-03-18T05:49:45Z)
• A framework for paired-sample hypothesis testing for high-dimensional data [7.400168551191579]
  We put forward the idea that scoring functions can be produced by the decision rules defined by the bisecting hyperplanes of the line segments connecting each pair of instances. First, we estimate the bisecting hyperplanes for each pair of instances and an aggregated rule derived through the Hodges-Lehmann estimator.
  arXiv  Detail & Related papers  (2023-09-28T09:17:11Z)
• Active Sequential Two-Sample Testing [18.99517340397671]
  We consider the two-sample testing problem in a new scenario where sample measurements are inexpensive to access. We devise the first emphactiveNIST-sample testing framework that not only sequentially but also emphactively queries. In practice, we introduce an instantiation of our framework and evaluate it using several experiments.
  arXiv  Detail & Related papers  (2023-01-30T02:23:49Z)
• Sequential Kernelized Independence Testing [101.22966794822084]
  We design sequential kernelized independence tests inspired by kernelized dependence measures. We demonstrate the power of our approaches on both simulated and real data.
  arXiv  Detail & Related papers  (2022-12-14T18:08:42Z)
• Learning to Increase the Power of Conditional Randomization Tests [8.883733362171032]
  The model-X conditional randomization test is a generic framework for conditional independence testing. We introduce novel model-fitting schemes that are designed to explicitly improve the power of model-X tests.
  arXiv  Detail & Related papers  (2022-07-03T12:29:25Z)
• Nonparametric Conditional Local Independence Testing [69.31200003384122]
  Conditional local independence is an independence relation among continuous time processes. No nonparametric test of conditional local independence has been available. We propose such a nonparametric test based on double machine learning.
  arXiv  Detail & Related papers  (2022-03-25T10:31:02Z)
• Training on Test Data with Bayesian Adaptation for Covariate Shift [96.3250517412545]
  Deep neural networks often make inaccurate predictions with unreliable uncertainty estimates. We derive a Bayesian model that provides for a well-defined relationship between unlabeled inputs under distributional shift and model parameters. We show that our method improves both accuracy and uncertainty estimation.
  arXiv  Detail & Related papers  (2021-09-27T01:09:08Z)
• Nonparametric Empirical Bayes Estimation and Testing for Sparse and Heteroscedastic Signals [5.715675926089834]
  Large-scale modern data often involves estimation and testing for high-dimensional unknown parameters. It is desirable to identify the sparse signals, the needles in the haystack'', with accuracy and false discovery control. We propose a novel Spike-and-Nonparametric mixture prior (SNP) -- a spike to promote the sparsity and a nonparametric structure to capture signals.
  arXiv  Detail & Related papers  (2021-06-16T15:55:44Z)
• Anomaly Detection of Time Series with Smoothness-Inducing Sequential Variational Auto-Encoder [59.69303945834122]
  We present a Smoothness-Inducing Sequential Variational Auto-Encoder (SISVAE) model for robust estimation and anomaly detection of time series. Our model parameterizes mean and variance for each time-stamp with flexible neural networks. We show the effectiveness of our model on both synthetic datasets and public real-world benchmarks.
  arXiv  Detail & Related papers  (2021-02-02T06:15:15Z)
• Ambiguity in Sequential Data: Predicting Uncertain Futures with Recurrent Models [110.82452096672182]
  We propose an extension of the Multiple Hypothesis Prediction (MHP) model to handle ambiguous predictions with sequential data. We also introduce a novel metric for ambiguous problems, which is better suited to account for uncertainties.
  arXiv  Detail & Related papers  (2020-03-10T09:15:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.