Noise-Contrastive Estimation for Multivariate Point Processes

• URL: http://arxiv.org/abs/2011.00717v1
• Date: Mon, 2 Nov 2020 04:09:33 GMT
• Title: Noise-Contrastive Estimation for Multivariate Point Processes
• Authors: Hongyuan Mei, Tom Wan, Jason Eisner
• Abstract summary: We show how to apply a noise-contrastive estimation method with a less expensive objective. For the model to achieve the same level of log-likelihood on held-out data, our method needs considerably fewer function evaluations and less wall-clock time.
• Score: 28.23193933174945
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The log-likelihood of a generative model often involves both positive and negative terms. For a temporal multivariate point process, the negative term sums over all the possible event types at each time and also integrates over all the possible times. As a result, maximum likelihood estimation is expensive. We show how to instead apply a version of noise-contrastive estimation---a general parameter estimation method with a less expensive stochastic objective. Our specific instantiation of this general idea works out in an interestingly non-trivial way and has provable guarantees for its optimality, consistency and efficiency. On several synthetic and real-world datasets, our method shows benefits: for the model to achieve the same level of log-likelihood on held-out data, our method needs considerably fewer function evaluations and less wall-clock time.

Related papers

• One Sample Fits All: Approximating All Probabilistic Values Simultaneously and Efficiently [19.265709097637643]
  Probable values have gained recent attention in applications like feature attribution and data valuation. We propose a one-sample-fits-all framework parameterized by a sampling vector to approximate intermediate terms. We show that our one-for-all estimator achieves the fastest convergence rate on Beta Shapley values.
  arXiv  Detail & Related papers  (2024-10-31T10:47:46Z)
• Cumulative Hazard Function Based Efficient Multivariate Temporal Point Process Learning [0.0]
  In this paper, we explore using neural networks to model a flexible but well-defined CHF. We show that the proposed model achieves the state-of-the-art performance on data fitting and event prediction tasks.
  arXiv  Detail & Related papers  (2024-04-21T13:51:31Z)
• Stochastic Amortization: A Unified Approach to Accelerate Feature and Data Attribution [62.71425232332837]
  We show that training amortized models with noisy labels is inexpensive and surprisingly effective. This approach significantly accelerates several feature attribution and data valuation methods, often yielding an order of magnitude speedup over existing approaches.
  arXiv  Detail & Related papers  (2024-01-29T03:42:37Z)
• Learning Unnormalized Statistical Models via Compositional Optimization [73.30514599338407]
  Noise-contrastive estimation(NCE) has been proposed by formulating the objective as the logistic loss of the real data and the artificial noise. In this paper, we study it a direct approach for optimizing the negative log-likelihood of unnormalized models.
  arXiv  Detail & Related papers  (2023-06-13T01:18:16Z)
• Active Sampling of Multiple Sources for Sequential Estimation [92.37271004438406]
  The objective is to design an active sampling algorithm for sequentially estimating parameters in order to form reliable estimates. This paper adopts emph conditional estimation cost functions, leading to a sequential estimation approach that was recently shown to render tractable analysis.
  arXiv  Detail & Related papers  (2022-08-10T15:58:05Z)
• Reliability analysis of discrete-state performance functions via adaptive sequential sampling with detection of failure surfaces [0.0]
  The paper presents a new efficient and robust method for rare event probability estimation. The method can estimate the probabilities of multiple failure types. It can accommodate this information to increase the accuracy of the estimated probabilities.
  arXiv  Detail & Related papers  (2022-08-04T05:59:25Z)
• Local policy search with Bayesian optimization [73.0364959221845]
  Reinforcement learning aims to find an optimal policy by interaction with an environment. Policy gradients for local search are often obtained from random perturbations. We develop an algorithm utilizing a probabilistic model of the objective function and its gradient.
  arXiv  Detail & Related papers  (2021-06-22T16:07:02Z)
• Scalable Personalised Item Ranking through Parametric Density Estimation [53.44830012414444]
  Learning from implicit feedback is challenging because of the difficult nature of the one-class problem. Most conventional methods use a pairwise ranking approach and negative samplers to cope with the one-class problem. We propose a learning-to-rank approach, which achieves convergence speed comparable to the pointwise counterpart.
  arXiv  Detail & Related papers  (2021-05-11T03:38:16Z)
• Time-varying Gaussian Process Bandit Optimization with Non-constant Evaluation Time [93.6788993843846]
  We propose a novel time-varying Bayesian optimization algorithm that can effectively handle the non-constant evaluation time. Our bound elucidates that a pattern of the evaluation time sequence can hugely affect the difficulty of the problem.
  arXiv  Detail & Related papers  (2020-03-10T13:28:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.