Hypothesis Testing for High-Dimensional Matrix-Valued Data

• URL: http://arxiv.org/abs/2412.07987v1
• Date: Tue, 10 Dec 2024 23:59:35 GMT
• Title: Hypothesis Testing for High-Dimensional Matrix-Valued Data
• Authors: Shijie Cui, Danning Li, Runze Li, Lingzhou Xue,
• Abstract summary: We investigate the minimum discrepancy test, originally proposed by Cragg (1997), which serves as a rank test for lower-dimensional matrices. To address these challenges, we propose a new test statistic tailored for high-dimensional matrix rank testing. The paper concludes with simulation studies and two case studies involving surveillance video data, demonstrating the practical utility of our proposed methods.
• Score: 7.622217735434662
• License:
• Abstract: This paper addresses hypothesis testing for the mean of matrix-valued data in high-dimensional settings. We investigate the minimum discrepancy test, originally proposed by Cragg (1997), which serves as a rank test for lower-dimensional matrices. We evaluate the performance of this test as the matrix dimensions increase proportionally with the sample size, and identify its limitations when matrix dimensions significantly exceed the sample size. To address these challenges, we propose a new test statistic tailored for high-dimensional matrix rank testing. The oracle version of this statistic is analyzed to highlight its theoretical properties. Additionally, we develop a novel approach for constructing a sparse singular value decomposition (SVD) estimator for singular vectors, providing a comprehensive examination of its theoretical aspects. Using the sparse SVD estimator, we explore the properties of the sample version of our proposed statistic. The paper concludes with simulation studies and two case studies involving surveillance video data, demonstrating the practical utility of our proposed methods.

Related papers

• Diffusion posterior sampling for simulation-based inference in tall data settings [53.17563688225137]
  Simulation-based inference ( SBI) is capable of approximating the posterior distribution that relates input parameters to a given observation. In this work, we consider a tall data extension in which multiple observations are available to better infer the parameters of the model. We compare our method to recently proposed competing approaches on various numerical experiments and demonstrate its superiority in terms of numerical stability and computational cost.
  arXiv  Detail & Related papers  (2024-04-11T09:23:36Z)
• Statistical Inference For Noisy Matrix Completion Incorporating Auxiliary Information [3.9748528039819977]
  This paper investigates statistical inference for noisy matrix completion in a semi-supervised model. We apply an iterative least squares (LS) estimation approach in our considered context. We show that our method only needs a few iterations, and the resulting entry-wise estimators of the low-rank matrix and the coefficient matrix are guaranteed to have normal distributions.
  arXiv  Detail & Related papers  (2024-03-22T01:06:36Z)
• Sparse PCA with Oracle Property [115.72363972222622]
  We propose a family of estimators based on the semidefinite relaxation of sparse PCA with novel regularizations. We prove that, another estimator within the family achieves a sharper statistical rate of convergence than the standard semidefinite relaxation of sparse PCA.
  arXiv  Detail & Related papers  (2023-12-28T02:52:54Z)
• A Generalized Latent Factor Model Approach to Mixed-data Matrix Completion with Entrywise Consistency [3.299672391663527]
  Matrix completion is a class of machine learning methods that concerns the prediction of missing entries in a partially observed matrix. We formulate it as a low-rank matrix estimation problem under a general family of non-linear factor models. We propose entrywise consistent estimators for estimating the low-rank matrix.
  arXiv  Detail & Related papers  (2022-11-17T00:24:47Z)
• Optimal Discriminant Analysis in High-Dimensional Latent Factor Models [1.4213973379473654]
  In high-dimensional classification problems, a commonly used approach is to first project the high-dimensional features into a lower dimensional space. We formulate a latent-variable model with a hidden low-dimensional structure to justify this two-step procedure. We propose a computationally efficient classifier that takes certain principal components (PCs) of the observed features as projections.
  arXiv  Detail & Related papers  (2022-10-23T21:45:53Z)
• Test Set Sizing Via Random Matrix Theory [91.3755431537592]
  This paper uses techniques from Random Matrix Theory to find the ideal training-testing data split for a simple linear regression. It defines "ideal" as satisfying the integrity metric, i.e. the empirical model error is the actual measurement noise. This paper is the first to solve for the training and test size for any model in a way that is truly optimal.
  arXiv  Detail & Related papers  (2021-12-11T13:18:33Z)
• Estimating Graph Dimension with Cross-validated Eigenvalues [5.0013150536632995]
  In applied statistics, estimating the number of latent dimensions or the number of clusters is a fundamental and recurring problem. We provide a cross-validated eigenvalues approach to this problem. We prove that our procedure consistently estimates $k$ in scenarios where all $k$ dimensions can be estimated.
  arXiv  Detail & Related papers  (2021-08-06T23:52:30Z)
• Generalized Multivariate Signs for Nonparametric Hypothesis Testing in High Dimensions [4.24243593213882]
  We show that tests using generalized signs display higher power than existing tests, while maintaining nominal type-I error rates. We provide example applications on the MNIST and Minnesota Twin Studies genomic data.
  arXiv  Detail & Related papers  (2021-07-02T14:31:44Z)
• Intrinsic Dimension Estimation [92.87600241234344]
  We introduce a new estimator of the intrinsic dimension and provide finite sample, non-asymptotic guarantees. We then apply our techniques to get new sample complexity bounds for Generative Adversarial Networks (GANs) depending on the intrinsic dimension of the data.
  arXiv  Detail & Related papers  (2021-06-08T00:05:39Z)
• Sparse PCA via $l_{2,p}$-Norm Regularization for Unsupervised Feature Selection [138.97647716793333]
  We propose a simple and efficient unsupervised feature selection method, by combining reconstruction error with $l_2,p$-norm regularization. We present an efficient optimization algorithm to solve the proposed unsupervised model, and analyse the convergence and computational complexity of the algorithm theoretically.
  arXiv  Detail & Related papers  (2020-12-29T04:08:38Z)
• Understanding Implicit Regularization in Over-Parameterized Single Index Model [55.41685740015095]
  We design regularization-free algorithms for the high-dimensional single index model. We provide theoretical guarantees for the induced implicit regularization phenomenon.
  arXiv  Detail & Related papers  (2020-07-16T13:27:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.