Data-driven Modeling for Distribution Grids Under Partial Observability

• URL: http://arxiv.org/abs/2108.08350v1
• Date: Wed, 18 Aug 2021 18:50:14 GMT
• Title: Data-driven Modeling for Distribution Grids Under Partial Observability
• Abstract summary: This paper addresses the partial observability issue of data-driven distribution modeling. Inspired by the sparse changes in residential loads, we advocate to regularize the group sparsity of the unobservable injections. Numerical results using real-world load data on the single-phase equivalent of the IEEE 123-bus test case have demonstrated the accuracy improvements.
• Score: 5.815007821143811
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Accurately modeling power distribution grids is crucial for designing effective monitoring and decision making algorithms. This paper addresses the partial observability issue of data-driven distribution modeling in order to improve the accuracy of line parameter estimation. Inspired by the sparse changes in residential loads, we advocate to regularize the group sparsity of the unobservable injections in a bi-linear estimation problem. The alternating minimization scheme of guaranteed convergence is proposed to take advantage of convex subproblems with efficient solutions. Numerical results using real-world load data on the single-phase equivalent of the IEEE 123-bus test case have demonstrated the accuracy improvements of the proposed solution over existing work for both parameter estimation and voltage modeling.

Related papers

• The Regularization Parameter: Sparse Precision Matrix Estimation [1.3532832187980637]
  Sparse precision matrix estimation provides an efficient framework for modeling conditional dependencies in high-dimensional, low-sample-size data.<n>A recurring challenge is appropriately selecting the regularization parameter that controls estimator sparsity.<n>We propose a closed-form, matrix-valued regularization parameter derived from the sampling distribution of the first-order optimality conditions.
  arXiv  Detail & Related papers  (2026-07-07T13:19:19Z)
• Joint Model and Data Sparsification via the Marginal Likelihood [53.29070892356214]
  We propose jointly learning individual feature and sample relevancies, enabling simultaneous model and data sparsification.<n>This symmetric pruning of model and data offers a natural extension that preserves conjugacy.<n> Empirical results across diverse regression tasks affirm that a joint ARD approach consistently yields both sparse and robust prediction models.
  arXiv  Detail & Related papers  (2026-05-28T13:26:53Z)
• Efficient Covariance Estimation for Sparsified Functional Data [51.69796254617083]
  proposed Random-knots (Random-knots-Spatial) and B-spline (Bspline-Spatial) estimators of the covariance function are computationally efficient.<n>Asymptotic pointwise of the covariance are obtained for sparsified individual trajectories under some regularity conditions.
  arXiv  Detail & Related papers  (2025-11-23T00:50:33Z)
• Efficient Group Lasso Regularized Rank Regression with Data-Driven Parameter Determination [2.847099287022546]
  High-dimensional regression often suffers from heavy-tailed noise and outliers, which can severely undermine the reliability of least-squares based methods.<n>To improve robustness, we adopt a non-smooth Wilcoxon score based rank objective and incorporate structured group sparsity regularization.<n>We also introduce a data-driven, simulation-based tuning rule and further establish a finite-sample error bound for the resulting estimator.
  arXiv  Detail & Related papers  (2025-10-13T15:45:58Z)
• Distributionally Robust Optimization with Adversarial Data Contamination [49.89480853499918]
  We focus on optimizing Wasserstein-1 DRO objectives for generalized linear models with convex Lipschitz loss functions.<n>Our primary contribution lies in a novel modeling framework that integrates robustness against training data contamination with robustness against distributional shifts.<n>This work establishes the first rigorous guarantees, supported by efficient computation, for learning under the dual challenges of data contamination and distributional shifts.
  arXiv  Detail & Related papers  (2025-07-14T18:34:10Z)
• Robust Spatiotemporal Epidemic Modeling with Integrated Adaptive Outlier Detection [7.5504472850103435]
  In epidemic modeling, outliers can distort parameter estimation and lead to misguided public health decisions.<n>We introduce a robust generalized additive model (RST-GAM) to mitigate this distortion.<n>We demonstrate the practical utility of RST-GAM by analyzing county-level COVID-19 infection data in the United States.
  arXiv  Detail & Related papers  (2025-07-12T19:23:25Z)
• Stochastic Optimization with Optimal Importance Sampling [49.484190237840714]
  We propose an iterative-based algorithm that jointly updates the decision and the IS distribution without requiring time-scale separation between the two. Our method achieves the lowest possible variable variance and guarantees global convergence under convexity of the objective and mild assumptions on the IS distribution family.
  arXiv  Detail & Related papers  (2025-04-04T16:10:18Z)
• Partial Transportability for Domain Generalization [56.37032680901525]
  Building on the theory of partial identification and transportability, this paper introduces new results for bounding the value of a functional of the target distribution. Our contribution is to provide the first general estimation technique for transportability problems. We propose a gradient-based optimization scheme for making scalable inferences in practice.
  arXiv  Detail & Related papers  (2025-03-30T22:06:37Z)
• Distributionally Robust Optimization as a Scalable Framework to Characterize Extreme Value Distributions [22.765095010254118]
  The goal of this paper is to develop distributionally robust optimization (DRO) estimators, specifically for multidimensional Extreme Value Theory (EVT) statistics. In order to mitigate over-conservative estimates while enhancing out-of-sample performance, we study DRO estimators informed by semi-parametric max-stable constraints in the space of point processes. Both approaches are validated using synthetically generated data, recovering prescribed characteristics, and verifying the efficacy of the proposed techniques.
  arXiv  Detail & Related papers  (2024-07-31T19:45:27Z)
• Variational Bayesian surrogate modelling with application to robust design optimisation [0.9626666671366836]
  Surrogate models provide a quick-to-evaluate approximation to complex computational models. We consider Bayesian inference for constructing statistical surrogates with input uncertainties and dimensionality reduction. We demonstrate intrinsic and robust structural optimisation problems where cost functions depend on a weighted sum of the mean and standard deviation of model outputs.
  arXiv  Detail & Related papers  (2024-04-23T09:22:35Z)
• Learning Graphical Factor Models with Riemannian Optimization [70.13748170371889]
  This paper proposes a flexible algorithmic framework for graph learning under low-rank structural constraints. The problem is expressed as penalized maximum likelihood estimation of an elliptical distribution. We leverage geometries of positive definite matrices and positive semi-definite matrices of fixed rank that are well suited to elliptical models.
  arXiv  Detail & Related papers  (2022-10-21T13:19:45Z)
• Distributed Estimation and Inference for Semi-parametric Binary Response Models [8.309294338998539]
  This paper studies the maximum score estimator of a semi-parametric binary choice model under a distributed computing environment. An intuitive divide-and-conquer estimator is computationally expensive and restricted by a non-regular constraint on the number of machines.
  arXiv  Detail & Related papers  (2022-10-15T23:06:46Z)
• Numerically Stable Sparse Gaussian Processes via Minimum Separation using Cover Trees [57.67528738886731]
  We study the numerical stability of scalable sparse approximations based on inducing points. For low-dimensional tasks such as geospatial modeling, we propose an automated method for computing inducing points satisfying these conditions.
  arXiv  Detail & Related papers  (2022-10-14T15:20:17Z)
• Distributed Sketching for Randomized Optimization: Exact Characterization, Concentration and Lower Bounds [54.51566432934556]
  We consider distributed optimization methods for problems where forming the Hessian is computationally challenging. We leverage randomized sketches for reducing the problem dimensions as well as preserving privacy and improving straggler resilience in asynchronous distributed systems.
  arXiv  Detail & Related papers  (2022-03-18T05:49:13Z)
• Support estimation in high-dimensional heteroscedastic mean regression [2.28438857884398]
  We consider a linear mean regression model with random design and potentially heteroscedastic, heavy-tailed errors. We use a strictly convex, smooth variant of the Huber loss function with tuning parameter depending on the parameters of the problem. For the resulting estimator we show sign-consistency and optimal rates of convergence in the $ell_infty$ norm.
  arXiv  Detail & Related papers  (2020-11-03T09:46:31Z)
• Robust Bayesian Inference for Discrete Outcomes with the Total Variation Distance [5.139874302398955]
  Models of discrete-valued outcomes are easily misspecified if the data exhibit zero-inflation, overdispersion or contamination. Here, we introduce a robust discrepancy-based Bayesian approach using the Total Variation Distance (TVD) We empirically demonstrate that our approach is robust and significantly improves predictive performance on a range of simulated and real world data.
  arXiv  Detail & Related papers  (2020-10-26T09:53:06Z)
• Unlabelled Data Improves Bayesian Uncertainty Calibration under Covariate Shift [100.52588638477862]
  We develop an approximate Bayesian inference scheme based on posterior regularisation. We demonstrate the utility of our method in the context of transferring prognostic models of prostate cancer across globally diverse populations.
  arXiv  Detail & Related papers  (2020-06-26T13:50:19Z)
• Efficient Ensemble Model Generation for Uncertainty Estimation with Bayesian Approximation in Segmentation [74.06904875527556]
  We propose a generic and efficient segmentation framework to construct ensemble segmentation models. In the proposed method, ensemble models can be efficiently generated by using the layer selection method. We also devise a new pixel-wise uncertainty loss, which improves the predictive performance.
  arXiv  Detail & Related papers  (2020-05-21T16:08:38Z)
• Distributed Averaging Methods for Randomized Second Order Optimization [54.51566432934556]
  We consider distributed optimization problems where forming the Hessian is computationally challenging and communication is a bottleneck. We develop unbiased parameter averaging methods for randomized second order optimization that employ sampling and sketching of the Hessian. We also extend the framework of second order averaging methods to introduce an unbiased distributed optimization framework for heterogeneous computing systems.
  arXiv  Detail & Related papers  (2020-02-16T09:01:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.