Finite-Sample Identification of Linear Regression Models with Residual-Permuted Sums

• URL: http://arxiv.org/abs/2406.05440v1
• Date: Sat, 8 Jun 2024 11:09:30 GMT
• Title: Finite-Sample Identification of Linear Regression Models with Residual-Permuted Sums
• Authors: Szabolcs Szentpéteri, Balázs Csanád Csáji,
• Abstract summary: Residual-Permuted Sums (RPS) is an alternative to the Sign-Perturbed Sums (SPS) algorithm, to construct confidence regions. RPS permutes the residuals instead of perturbing their signs. We provide the first proof that these permutation-based confidence regions are uniformly strongly consistent under general assumptions.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: This letter studies a distribution-free, finite-sample data perturbation (DP) method, the Residual-Permuted Sums (RPS), which is an alternative of the Sign-Perturbed Sums (SPS) algorithm, to construct confidence regions. While SPS assumes independent (but potentially time-varying) noise terms which are symmetric about zero, RPS gets rid of the symmetricity assumption, but assumes i.i.d. noises. The main idea is that RPS permutes the residuals instead of perturbing their signs. This letter introduces RPS in a flexible way, which allows various design-choices. RPS has exact finite sample coverage probabilities and we provide the first proof that these permutation-based confidence regions are uniformly strongly consistent under general assumptions. This means that the RPS regions almost surely shrink around the true parameters as the sample size increases. The ellipsoidal outer-approximation (EOA) of SPS is also extended to RPS, and the effectiveness of RPS is validated by numerical experiments, as well.

Related papers

• Reliability Assessment of Information Sources Based on Random Permutation Set [9.542461785588925]
  In pattern recognition, handling uncertainty is a critical challenge that significantly affects decision-making and classification accuracy. There is a lack of a transformation method based on permutation order between Random Permutation Set (RPS) and Dempster-Shafer Theory (DST) This paper proposes an RPS transformation approach and a probability transformation method tailored for RPS.
  arXiv  Detail & Related papers  (2024-10-30T07:40:35Z)
• Finite Sample Analysis of Distribution-Free Confidence Ellipsoids for Linear Regression [0.0]
  The least squares (LS) estimate is the archetypical solution of linear regression problems. The paper studies the distribution-free Sign-Perturbed Sums (SPS) ellipsoidal outer approximation (EOA) algorithm which can construct non-asymptotically guaranteed confidence ellipsoids.
  arXiv  Detail & Related papers  (2024-09-13T13:10:02Z)
• Sample Complexity of the Sign-Perturbed Sums Method [0.0]
  The Sign-Perturbed Sums (SPS) method constructs exact, non-asymptotic confidence regions for the true system parameters. We study the sample complexity of SPS for general linear regression problems. The difference between the theoretical bounds and the empirical sizes of SPS confidence regions is investigated experimentally.
  arXiv  Detail & Related papers  (2024-09-02T13:18:53Z)
• Relaxed Quantile Regression: Prediction Intervals for Asymmetric Noise [51.87307904567702]
  Quantile regression is a leading approach for obtaining such intervals via the empirical estimation of quantiles in the distribution of outputs. We propose Relaxed Quantile Regression (RQR), a direct alternative to quantile regression based interval construction that removes this arbitrary constraint. We demonstrate that this added flexibility results in intervals with an improvement in desirable qualities.
  arXiv  Detail & Related papers  (2024-06-05T13:36:38Z)
• Sample Complexity of the Sign-Perturbed Sums Identification Method: Scalar Case [0.0]
  Sign-Perturbed Sum (SPS) is a powerful finite-sample system identification algorithm. This paper studies the behaviour of SPS confidence intervals.
  arXiv  Detail & Related papers  (2024-01-28T22:44:41Z)
• Robustness of optimized numerical estimation schemes for noisy variational quantum algorithms [0.0]
  We explore the extent to which numerical schemes remain statistically more accurate for a given number of sampling copies in the presence of noise. For noise-channel error terms that are independent of the circuit parameters, we demonstrate that emph without any knowledge about the noise channel. We show that these optimized SPS estimators can significantly reduce mean-squared-error biases.
  arXiv  Detail & Related papers  (2023-10-07T08:43:26Z)
• Policy Evaluation in Distributional LQR [70.63903506291383]
  We provide a closed-form expression of the distribution of the random return. We show that this distribution can be approximated by a finite number of random variables. Using the approximate return distribution, we propose a zeroth-order policy gradient algorithm for risk-averse LQR.
  arXiv  Detail & Related papers  (2023-03-23T20:27:40Z)
• Training Discrete Deep Generative Models via Gapped Straight-Through Estimator [72.71398034617607]
  We propose a Gapped Straight-Through ( GST) estimator to reduce the variance without incurring resampling overhead. This estimator is inspired by the essential properties of Straight-Through Gumbel-Softmax. Experiments demonstrate that the proposed GST estimator enjoys better performance compared to strong baselines on two discrete deep generative modeling tasks.
  arXiv  Detail & Related papers  (2022-06-15T01:46:05Z)
• Sampling-Based Robust Control of Autonomous Systems with Non-Gaussian Noise [59.47042225257565]
  We present a novel planning method that does not rely on any explicit representation of the noise distributions. First, we abstract the continuous system into a discrete-state model that captures noise by probabilistic transitions between states. We capture these bounds in the transition probability intervals of a so-called interval Markov decision process (iMDP)
  arXiv  Detail & Related papers  (2021-10-25T06:18:55Z)
• Improved, Deterministic Smoothing for L1 Certified Robustness [119.86676998327864]
  We propose a non-additive and deterministic smoothing method, Deterministic Smoothing with Splitting Noise (DSSN) In contrast to uniform additive smoothing, the SSN certification does not require the random noise components used to be independent. This is the first work to provide deterministic "randomized smoothing" for a norm-based adversarial threat model.
  arXiv  Detail & Related papers  (2021-03-17T21:49:53Z)
• Modal Regression based Structured Low-rank Matrix Recovery for Multi-view Learning [70.57193072829288]
  Low-rank Multi-view Subspace Learning has shown great potential in cross-view classification in recent years. Existing LMvSL based methods are incapable of well handling view discrepancy and discriminancy simultaneously. We propose Structured Low-rank Matrix Recovery (SLMR), a unique method of effectively removing view discrepancy and improving discriminancy.
  arXiv  Detail & Related papers  (2020-03-22T03:57:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.