Neural Quasiprobabilistic Likelihood Ratio Estimation with Negatively Weighted Data

• URL: http://arxiv.org/abs/2410.10216v1
• Date: Mon, 14 Oct 2024 07:15:49 GMT
• Title: Neural Quasiprobabilistic Likelihood Ratio Estimation with Negatively Weighted Data
• Abstract summary: We consider a generalisation of the likelihood-ratio estimation task to a quasiprobabilistic setting where probability densities can be negative. The presence of negative densities and negative weights, pose an array of challenges to traditional neural likelihood ratio estimation methods.
• Score: 2.5923147161639104
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Motivated by real-world situations found in high energy particle physics, we consider a generalisation of the likelihood-ratio estimation task to a quasiprobabilistic setting where probability densities can be negative. By extension, this framing also applies to importance sampling in a setting where the importance weights can be negative. The presence of negative densities and negative weights, pose an array of challenges to traditional neural likelihood ratio estimation methods. We address these challenges by introducing a novel loss function. In addition, we introduce a new model architecture based on the decomposition of a likelihood ratio using signed mixture models, providing a second strategy for overcoming these challenges. Finally, we demonstrate our approach on a pedagogical example and a real-world example from particle physics.

Related papers

• A Convex Approximation Framework for Neural Likelihood-Based Bayesian Inverse Problems [0.0]
  We study neural likelihood approximation, where the goal is to learn the likelihood function directly from data without explicit knowledge of the underlying data-generating process.<n>We show that empirical minimizers of the resulting data-driven objective converge to the true likelihood as the sample size grows.
  arXiv  Detail & Related papers  (2026-07-07T13:18:45Z)
• Model-agnostic Mitigation Strategies of Data Imbalance for Regression [0.0]
  Data imbalance persists as a pervasive challenge in regression tasks, introducing bias in model performance and undermining predictive reliability.<n>We present advanced mitigation techniques, which build upon and improve existing sampling methods.<n>We demonstrate that constructing an ensemble of models -- one trained with imbalance mitigation and another without -- can significantly reduce these negative effects.
  arXiv  Detail & Related papers  (2025-06-02T09:46:08Z)
• Rethinking Diffusion Model in High Dimension [0.0]
  Diffusion models assume that they can learn the statistical quantities of the underlying probability distribution.<n>But is this really how they work?<n>Most inference methods can be unified within a simple framework.
  arXiv  Detail & Related papers  (2025-03-11T17:36:11Z)
• Rethinking negative sampling in content-based news recommendation [1.5416095780642964]
  News recommender systems are hindered by the brief lifespan of articles, as they undergo rapid relevance decay. Recent studies have demonstrated the potential of content-based neural techniques in tackling this problem. In this study, we posit that the careful sampling of negative examples has a big impact on the model's outcome.
  arXiv  Detail & Related papers  (2024-11-13T15:42:13Z)
• Estimating Causal Effects from Learned Causal Networks [56.14597641617531]
  We propose an alternative paradigm for answering causal-effect queries over discrete observable variables. We learn the causal Bayesian network and its confounding latent variables directly from the observational data. We show that this emphmodel completion learning approach can be more effective than estimand approaches.
  arXiv  Detail & Related papers  (2024-08-26T08:39:09Z)
• Embedding generalization within the learning dynamics: An approach based-on sample path large deviation theory [0.0]
  We consider an empirical risk perturbation based learning problem that exploits methods from continuous-time perspective. We provide an estimate in the small noise limit based on the Freidlin-Wentzell theory of large deviations. We also present a computational algorithm that solves the corresponding variational problem leading to an optimal point estimates.
  arXiv  Detail & Related papers  (2024-08-04T23:31:35Z)
• Scaling and renormalization in high-dimensional regression [72.59731158970894]
  We present a unifying perspective on recent results on ridge regression.<n>We use the basic tools of random matrix theory and free probability, aimed at readers with backgrounds in physics and deep learning.<n>Our results extend and provide a unifying perspective on earlier models of scaling laws.
  arXiv  Detail & Related papers  (2024-05-01T15:59:00Z)
• The Surprising Harmfulness of Benign Overfitting for Adversarial Robustness [13.120373493503772]
  We prove a surprising result that even if the ground truth itself is robust to adversarial examples, the benignly overfitted model is benign in terms of the standard'' out-of-sample risk objective. Our finding provides theoretical insights into the puzzling phenomenon observed in practice, where the true target function (e.g., human) is robust against adverasrial attack, while beginly overfitted neural networks lead to models that are not robust.
  arXiv  Detail & Related papers  (2024-01-19T15:40:46Z)
• Bias-inducing geometries: an exactly solvable data model with fairness implications [13.690313475721094]
  We introduce an exactly solvable high-dimensional model of data imbalance. We analytically unpack the typical properties of learning models trained in this synthetic framework. We obtain exact predictions for the observables that are commonly employed for fairness assessment.
  arXiv  Detail & Related papers  (2022-05-31T16:27:57Z)
• Adversarial Regression with Doubly Non-negative Weighting Matrices [25.733130388781476]
  We propose a novel and coherent scheme for kernel-reweighted regression by reparametrizing the sample weights using a doubly non-negative matrix. Numerical experiments show that our reweighting strategy delivers promising results on numerous datasets.
  arXiv  Detail & Related papers  (2021-09-30T06:41:41Z)
• The Interplay Between Implicit Bias and Benign Overfitting in Two-Layer Linear Networks [51.1848572349154]
  neural network models that perfectly fit noisy data can generalize well to unseen test data. We consider interpolating two-layer linear neural networks trained with gradient flow on the squared loss and derive bounds on the excess risk.
  arXiv  Detail & Related papers  (2021-08-25T22:01:01Z)
• Residual Error: a New Performance Measure for Adversarial Robustness [85.0371352689919]
  A major challenge that limits the wide-spread adoption of deep learning has been their fragility to adversarial attacks. This study presents the concept of residual error, a new performance measure for assessing the adversarial robustness of a deep neural network. Experimental results using the case of image classification demonstrate the effectiveness and efficacy of the proposed residual error metric.
  arXiv  Detail & Related papers  (2021-06-18T16:34:23Z)
• Contextual Dropout: An Efficient Sample-Dependent Dropout Module [60.63525456640462]
  Dropout has been demonstrated as a simple and effective module to regularize the training process of deep neural networks. We propose contextual dropout with an efficient structural design as a simple and scalable sample-dependent dropout module. Our experimental results show that the proposed method outperforms baseline methods in terms of both accuracy and quality of uncertainty estimation.
  arXiv  Detail & Related papers  (2021-03-06T19:30:32Z)
• Non-Singular Adversarial Robustness of Neural Networks [58.731070632586594]
  Adrial robustness has become an emerging challenge for neural network owing to its over-sensitivity to small input perturbations. We formalize the notion of non-singular adversarial robustness for neural networks through the lens of joint perturbations to data inputs as well as model weights.
  arXiv  Detail & Related papers  (2021-02-23T20:59:30Z)
• On the Transferability of Adversarial Attacksagainst Neural Text Classifier [121.6758865857686]
  We investigate the transferability of adversarial examples for text classification models. We propose a genetic algorithm to find an ensemble of models that can induce adversarial examples to fool almost all existing models. We derive word replacement rules that can be used for model diagnostics from these adversarial examples.
  arXiv  Detail & Related papers  (2020-11-17T10:45:05Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.