Scale-invariant Optimal Sampling for Rare-events Data with Sparse Models

• URL: http://arxiv.org/abs/2608.22597v1
• Date: Sun, 23 Aug 2026 21:03:34 GMT
• Title: Scale-invariant Optimal Sampling for Rare-events Data with Sparse Models
• Abstract summary: Subsampling is effective in tackling computational challenges for massive data with rare events.<n>Existing optimal subsampling probabilities depend on data scales, and some scaling transformations may result in inefficient subsamples.<n>We tackle this challenge and introduce a scale-invariant optimal subsampling function in the context of sparse models.
• Score: 9.971349070438356
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Subsampling is effective in tackling computational challenges for massive data with rare events. Overly aggressive subsampling may adversely affect estimation efficiency, and optimal subsampling is essential to mitigate the information loss. However, existing optimal subsampling probabilities depend on data scales, and some scaling transformations may result in inefficient subsamples. This problem is more significant when there are inactive features, because their influence on the subsampling probabilities can be arbitrarily magnified by inappropriate scaling transformations. We tackle this challenge and introduce a scale-invariant optimal subsampling function in the context of sparse models, where inactive features are commonly assumed. Instead of focusing on estimating model parameters, we define an optimal subsampling function to minimize the prediction error, using adaptive lasso to outline the estimation procedure and study its theoretical guarantee. We first introduce the adaptive lasso estimator for rare-events data and establish its oracle properties, thereby validating the use of subsampling. Then we derive a scale-invariant optimal subsampling function that minimizes the prediction error of the inverse probability weighted (IPW) adaptive lasso. Finally, we present an estimator based on the maximum sampled conditional likelihood (MSCL) to further improve the estimation efficiency. We conduct numerical experiments using both simulated and real-world data sets to demonstrate the performance of the proposed methods.

Related papers

• Robust Spatiotemporal Epidemic Modeling with Integrated Adaptive Outlier Detection [7.5504472850103435]
  In epidemic modeling, outliers can distort parameter estimation and lead to misguided public health decisions.<n>We introduce a robust generalized additive model (RST-GAM) to mitigate this distortion.<n>We demonstrate the practical utility of RST-GAM by analyzing county-level COVID-19 infection data in the United States.
  arXiv  Detail & Related papers  (2025-07-12T19:23:25Z)
• Hybrid least squares for learning functions from highly noisy data [7.096701481970196]
  We consider a least-squares function approximation problem with heavily polluted data.<n>Existing methods that are powerful in the small noise regime are suboptimal when large noise is present.<n>We show that the proposed algorithm enjoys appropriate optimality properties for both sample point generation and noise mollification.
  arXiv  Detail & Related papers  (2025-07-03T00:31:29Z)
• Optimal Sampling for Generalized Linear Model under Measurement Constraint with Surrogate Variables [3.5903555216741405]
  In some cases, surrogate variables are accessible across the entire dataset and can serve as approximations to the true response variable.<n>We propose an optimal sampling strategy that effectively harnesses the available information from surrogate variables.
  arXiv  Detail & Related papers  (2025-01-01T22:41:52Z)
• Accelerated zero-order SGD under high-order smoothness and overparameterized regime [79.85163929026146]
  We present a novel gradient-free algorithm to solve convex optimization problems. Such problems are encountered in medicine, physics, and machine learning. We provide convergence guarantees for the proposed algorithm under both types of noise.
  arXiv  Detail & Related papers  (2024-11-21T10:26:17Z)
• Risk-Sensitive Diffusion: Robustly Optimizing Diffusion Models with Noisy Samples [58.68233326265417]
  Non-image data are prevalent in real applications and tend to be noisy. Risk-sensitive SDE is a type of differential equation (SDE) parameterized by the risk vector. We conduct systematic studies for both Gaussian and non-Gaussian noise distributions.
  arXiv  Detail & Related papers  (2024-02-03T08:41:51Z)
• Self-Supervised Dataset Distillation for Transfer Learning [77.4714995131992]
  We propose a novel problem of distilling an unlabeled dataset into a set of small synthetic samples for efficient self-supervised learning (SSL) We first prove that a gradient of synthetic samples with respect to a SSL objective in naive bilevel optimization is textitbiased due to randomness originating from data augmentations or masking. We empirically validate the effectiveness of our method on various applications involving transfer learning.
  arXiv  Detail & Related papers  (2023-10-10T10:48:52Z)
• Fine-grained Retrieval Prompt Tuning [149.9071858259279]
  Fine-grained Retrieval Prompt Tuning steers a frozen pre-trained model to perform the fine-grained retrieval task from the perspectives of sample prompt and feature adaptation. Our FRPT with fewer learnable parameters achieves the state-of-the-art performance on three widely-used fine-grained datasets.
  arXiv  Detail & Related papers  (2022-07-29T04:10:04Z)
• Variational Inference with NoFAS: Normalizing Flow with Adaptive Surrogate for Computationally Expensive Models [7.217783736464403]
  Use of sampling-based approaches such as Markov chain Monte Carlo may become intractable when each likelihood evaluation is computationally expensive. New approaches combining variational inference with normalizing flow are characterized by a computational cost that grows only linearly with the dimensionality of the latent variable space. We propose Normalizing Flow with Adaptive Surrogate (NoFAS), an optimization strategy that alternatively updates the normalizing flow parameters and the weights of a neural network surrogate model.
  arXiv  Detail & Related papers  (2021-08-28T14:31:45Z)
• Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning [78.83598532168256]
  Marginal-likelihood based model-selection is rarely used in deep learning due to estimation difficulties. Our work shows that marginal likelihoods can improve generalization and be useful when validation data is unavailable.
  arXiv  Detail & Related papers  (2021-04-11T09:50:24Z)
• Maximum sampled conditional likelihood for informative subsampling [4.708378681950648]
  Subsampling is a computationally effective approach to extract information from massive data sets when computing resources are limited. We propose to use the maximum maximum conditional likelihood estimator (MSCLE) based on the sampled data.
  arXiv  Detail & Related papers  (2020-11-11T16:01:17Z)
• Slice Sampling for General Completely Random Measures [74.24975039689893]
  We present a novel Markov chain Monte Carlo algorithm for posterior inference that adaptively sets the truncation level using auxiliary slice variables. The efficacy of the proposed algorithm is evaluated on several popular nonparametric models.
  arXiv  Detail & Related papers  (2020-06-24T17:53:53Z)
• Optimal Distributed Subsampling for Maximum Quasi-Likelihood Estimators with Massive Data [20.79270369203348]
  Existing methods mostly focus on subsampling with replacement due to its high computational efficiency. We first derive optimal subsampling probabilities in the context of quasi-likelihood estimation. We develop a distributed subsampling framework, in which statistics are computed simultaneously on smaller partitions of the full data.
  arXiv  Detail & Related papers  (2020-05-21T02:46:56Z)
• SUMO: Unbiased Estimation of Log Marginal Probability for Latent Variable Models [80.22609163316459]
  We introduce an unbiased estimator of the log marginal likelihood and its gradients for latent variable models based on randomized truncation of infinite series. We show that models trained using our estimator give better test-set likelihoods than a standard importance-sampling based approach for the same average computational cost.
  arXiv  Detail & Related papers  (2020-04-01T11:49:30Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.