Joint Estimation of Conditional Mean and Covariance for Unbalanced Panels

• URL: http://arxiv.org/abs/2410.21858v3
• Date: Thu, 14 Nov 2024 10:54:53 GMT
• Title: Joint Estimation of Conditional Mean and Covariance for Unbalanced Panels
• Authors: Damir Filipovic, Paul Schneider,
• Abstract summary: We propose a nonparametric, kernel-based joint estimator for conditional mean and covariance matrices in large unbalanced panels. Our estimator is applied to a comprehensive panel of monthly US stock excess returns from 1962 to 2021, conditioned on macroeconomic and firm-specific co variables. In asset pricing, it generates conditional mean-variance efficient portfolios with out-of-sample Sharpe ratios that substantially exceed those of equal-weighted benchmarks.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: We propose a nonparametric, kernel-based joint estimator for conditional mean and covariance matrices in large unbalanced panels. Our estimator, with proven consistency and finite-sample guarantees, is applied to a comprehensive panel of monthly US stock excess returns from 1962 to 2021, conditioned on macroeconomic and firm-specific covariates. The estimator captures time-varying cross-sectional dependencies effectively, demonstrating robust statistical performance. In asset pricing, it generates conditional mean-variance efficient portfolios with out-of-sample Sharpe ratios that substantially exceed those of equal-weighted benchmarks.

Related papers

• Online Policy Learning via a Self-Normalized Maximal Inequality [4.906641452356241]
  We develop a self-normalized maximal inequality for martingale empirical processes.<n>We show that, when combined with sequential updates and under standard complexity and margin conditions, the resulting estimator achieves fast convergence rates.
  arXiv  Detail & Related papers  (2025-10-17T09:53:42Z)
• Penalized Empirical Likelihood for Doubly Robust Causal Inference under Contamination in High Dimensions [0.720409153108429]
  We propose a doubly robust estimator for the average treatment effect in low sample size equations.<n>We show that the proposed confidence interval remain efficient compared to those competing estimates.
  arXiv  Detail & Related papers  (2025-07-23T11:58:54Z)
• Doubly-Robust Estimation of Counterfactual Policy Mean Embeddings [24.07815507403025]
  Estimating the distribution of outcomes under counterfactual policies is critical for decision-making in domains such as recommendation, advertising, and healthcare.<n>We analyze a novel framework-Counterfactual Policy Mean Embedding (CPME)-that represents the entire counterfactual outcome distribution in a reproducing kernel Hilbert space.
  arXiv  Detail & Related papers  (2025-06-03T12:16:46Z)
• A Distributionally-Robust Framework for Nuisance in Causal Effect Estimation [1.3053649021965603]
  Causal inference requires evaluating models on balanced distributions between treatment and control groups.<n>Most conventional statistical methods address this distribution shift through inverse probability weighting (IPW)<n>We decompose the error generalization to isolate these issues--propensity ambiguity and statistical instability--and address them through an adversarial loss function.
  arXiv  Detail & Related papers  (2025-05-23T10:34:28Z)
• Rectifying Conformity Scores for Better Conditional Coverage [75.73184036344908]
  We present a new method for generating confidence sets within the split conformal prediction framework. Our method performs a trainable transformation of any given conformity score to improve conditional coverage while ensuring exact marginal coverage.
  arXiv  Detail & Related papers  (2025-02-22T19:54:14Z)
• Benign Overfitting in Out-of-Distribution Generalization of Linear Models [19.203753135860016]
  We take an initial step towards understanding benign overfitting in the Out-of-Distribution (OOD) regime. We provide non-asymptotic guarantees proving that benign overfitting occurs in standard ridge regression. We also present theoretical results for a more general family of target covariance matrix.
  arXiv  Detail & Related papers  (2024-12-19T02:47:39Z)
• Bridging Fairness Gaps: A (Conditional) Distance Covariance Perspective in Fairness Learning [0.0]
  We bridge fairness gaps from a statistical perspective by selectively utilizing distance covariance statistics. We enhance fairness by incorporating sample (conditional) distance covariance as a manageable penalty term into the machine learning process.
  arXiv  Detail & Related papers  (2024-12-01T08:01:19Z)
• Probabilistic Conformal Prediction with Approximate Conditional Validity [81.30551968980143]
  We develop a new method for generating prediction sets that combines the flexibility of conformal methods with an estimate of the conditional distribution. Our method consistently outperforms existing approaches in terms of conditional coverage.
  arXiv  Detail & Related papers  (2024-07-01T20:44:48Z)
• Contextual Optimization under Covariate Shift: A Robust Approach by Intersecting Wasserstein Balls [18.047245099229325]
  We propose a distributionally robust approach that uses an ambiguity set by the intersection of two Wasserstein balls. We demonstrate the strong empirical performance of our proposed models.
  arXiv  Detail & Related papers  (2024-06-04T15:46:41Z)
• Portfolio Optimization with Robust Covariance and Conditional Value-at-Risk Constraints [0.0]
  We evaluated the performance of large-cap portfolio using various forms of Ledoit Shrinkage Covariance and Robust Gerber Covariance matrix. robustness estimators can outperform the market capitalization-weighted benchmark portfolio, particularly during bull markets. We incorporated unsupervised clustering algorithm K-means to the optimization algorithm.
  arXiv  Detail & Related papers  (2024-06-02T03:50:20Z)
• A Geometric Unification of Distributionally Robust Covariance Estimators: Shrinking the Spectrum by Inflating the Ambiguity Set [20.166217494056916]
  We propose a principled approach to construct covariance estimators without imposing restrictive assumptions. We show that our robust estimators are efficiently computable and consistent. Numerical experiments based on synthetic and real data show that our robust estimators are competitive with state-of-the-art estimators.
  arXiv  Detail & Related papers  (2024-05-30T15:01:18Z)
• Kernel-based optimally weighted conformal prediction intervals [12.814084012624916]
  We present Kernel-based Optimally Weighted Conformal Prediction Intervals (KOWCPI) KOWCPI adapts the classic Reweighted Nadaraya-Watson (RNW) estimator for quantile regression on dependent data. We demonstrate the superior performance of KOWCPI on real time-series against state-of-the-art methods.
  arXiv  Detail & Related papers  (2024-05-27T04:49:41Z)
• Efficient and Sharp Off-Policy Evaluation in Robust Markov Decision Processes [44.974100402600165]
  We study the evaluation of a policy best-parametric and worst-case perturbations to a decision process (MDP) We use transition observations from the original MDP, whether they are generated under the same or a different policy. Our estimator is also estimated statistical inference using Wald confidence intervals.
  arXiv  Detail & Related papers  (2024-03-29T18:11:49Z)
• Sparse PCA with Oracle Property [115.72363972222622]
  We propose a family of estimators based on the semidefinite relaxation of sparse PCA with novel regularizations. We prove that, another estimator within the family achieves a sharper statistical rate of convergence than the standard semidefinite relaxation of sparse PCA.
  arXiv  Detail & Related papers  (2023-12-28T02:52:54Z)
• Forecasting Large Realized Covariance Matrices: The Benefits of Factor Models and Shrinkage [1.0323063834827415]
  We decompose the return covariance matrix using standard firm-level factors and use sectoral restrictions in the residual covariance matrix. Our methodology improves forecasting precision relative to standard benchmarks and leads to better estimates of minimum variance portfolios.
  arXiv  Detail & Related papers  (2023-03-22T16:38:22Z)
• Robustifying Markowitz [3.154269505086154]
  The heavy-tail characteristics of financial time series are in fact the cause for these erratic fluctuations of weights. We present a toolbox for stabilizing costs and weights for global minimum Markowitz portfolios. We demonstrate that robustified portfolios reach the lowest turnover compared to shrinkage-based and constrained portfolios.
  arXiv  Detail & Related papers  (2022-12-28T18:09:14Z)
• A Unified Framework for Multi-distribution Density Ratio Estimation [101.67420298343512]
  Binary density ratio estimation (DRE) provides the foundation for many state-of-the-art machine learning algorithms. We develop a general framework from the perspective of Bregman minimization divergence. We show that our framework leads to methods that strictly generalize their counterparts in binary DRE.
  arXiv  Detail & Related papers  (2021-12-07T01:23:20Z)
• Deconfounding Scores: Feature Representations for Causal Effect Estimation with Weak Overlap [140.98628848491146]
  We introduce deconfounding scores, which induce better overlap without biasing the target of estimation. We show that deconfounding scores satisfy a zero-covariance condition that is identifiable in observed data. In particular, we show that this technique could be an attractive alternative to standard regularizations.
  arXiv  Detail & Related papers  (2021-04-12T18:50:11Z)
• Comparing Probability Distributions with Conditional Transport [63.11403041984197]
  We propose conditional transport (CT) as a new divergence and approximate it with the amortized CT (ACT) cost. ACT amortizes the computation of its conditional transport plans and comes with unbiased sample gradients that are straightforward to compute. On a wide variety of benchmark datasets generative modeling, substituting the default statistical distance of an existing generative adversarial network with ACT is shown to consistently improve the performance.
  arXiv  Detail & Related papers  (2020-12-28T05:14:22Z)
• Beyond Individual and Group Fairness [90.4666341812857]
  We present a new data-driven model of fairness that is guided by the unfairness complaints received by the system. Our model supports multiple fairness criteria and takes into account their potential incompatibilities.
  arXiv  Detail & Related papers  (2020-08-21T14:14:44Z)
• Fair Regression with Wasserstein Barycenters [39.818025466204055]
  We study the problem of learning a real-valued function that satisfies the Demographic Parity constraint. It demands the distribution of the predicted output to be independent of the sensitive attribute. We establish a connection between fair regression and optimal transport theory, based on which we derive a close form expression for the optimal fair predictor.
  arXiv  Detail & Related papers  (2020-06-12T16:10:41Z)
• Machine learning for causal inference: on the use of cross-fit estimators [77.34726150561087]
  Doubly-robust cross-fit estimators have been proposed to yield better statistical properties. We conducted a simulation study to assess the performance of several estimators for the average causal effect (ACE) When used with machine learning, the doubly-robust cross-fit estimators substantially outperformed all of the other estimators in terms of bias, variance, and confidence interval coverage.
  arXiv  Detail & Related papers  (2020-04-21T23:09:55Z)
• Distributional Robustness and Regularization in Reinforcement Learning [62.23012916708608]
  We introduce a new regularizer for empirical value functions and show that it lower bounds the Wasserstein distributionally robust value function. It suggests using regularization as a practical tool for dealing with $textitexternal uncertainty$ in reinforcement learning.
  arXiv  Detail & Related papers  (2020-03-05T19:56:23Z)
• GenDICE: Generalized Offline Estimation of Stationary Values [108.17309783125398]
  We show that effective estimation can still be achieved in important applications. Our approach is based on estimating a ratio that corrects for the discrepancy between the stationary and empirical distributions. The resulting algorithm, GenDICE, is straightforward and effective.
  arXiv  Detail & Related papers  (2020-02-21T00:27:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.