MCNet: Monotonic Calibration Networks for Expressive Uncertainty Calibration in Online Advertising

• URL: http://arxiv.org/abs/2503.00334v1
• Date: Sat, 01 Mar 2025 03:54:58 GMT
• Title: MCNet: Monotonic Calibration Networks for Expressive Uncertainty Calibration in Online Advertising
• Abstract summary: In online advertising, uncertainty calibration aims to adjust a ranking model's probability predictions to better approximate the true likelihood of an event.<n>Existing calibration approaches may lack the ability to model complex nonlinear relations, consider context features, and achieve balanced performance across different data subsets.<n>We introduce a novel model called Monotonic Networks, featuring three key designs: a monotonic calibration function, an order-preserving regularizer, and a field-balance regularizer.
• Score: 27.08760230660561
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: In online advertising, uncertainty calibration aims to adjust a ranking model's probability predictions to better approximate the true likelihood of an event, e.g., a click or a conversion. However, existing calibration approaches may lack the ability to effectively model complex nonlinear relations, consider context features, and achieve balanced performance across different data subsets. To tackle these challenges, we introduce a novel model called Monotonic Calibration Networks, featuring three key designs: a monotonic calibration function (MCF), an order-preserving regularizer, and a field-balance regularizer. The nonlinear MCF is capable of naturally modeling and universally approximating the intricate relations between uncalibrated predictions and the posterior probabilities, thus being much more expressive than existing methods. MCF can also integrate context features using a flexible model architecture, thereby achieving context awareness. The order-preserving and field-balance regularizers promote the monotonic relationship between adjacent bins and the balanced calibration performance on data subsets, respectively. Experimental results on both public and industrial datasets demonstrate the superior performance of our method in generating well-calibrated probability predictions.

Related papers

• Robust Bayesian Inference for Unnormalized Models with Mixed-Domain Data [2.842028685390757]
  SME-BETEL is a semiparametric framework that combines score matching equations with exponentially tilted empirical likelihood.<n>We show that SME-BETEL credible sets are normalizingally calibrated to the sampling variability of the score matching estimator.<n>This construction enables robust Bayesian inference for mixed-domain doubly-intractable models.
  arXiv  Detail & Related papers  (2026-09-01T18:51:42Z)
• Joint Model and Data Sparsification via the Marginal Likelihood [53.29070892356214]
  We propose jointly learning individual feature and sample relevancies, enabling simultaneous model and data sparsification.<n>This symmetric pruning of model and data offers a natural extension that preserves conjugacy.<n> Empirical results across diverse regression tasks affirm that a joint ARD approach consistently yields both sparse and robust prediction models.
  arXiv  Detail & Related papers  (2026-05-28T13:26:53Z)
• Another Fit Bites the Dust: Conformal Prediction as a Calibration Standard for Machine Learning in High-Energy Physics [0.0]
  Conformal prediction provides a distribution-free framework for calibrating arbitrary predictive models.<n>We show that a single conformal formalism can be applied across regression, binary and multi-class classification, anomaly detection, and generative modelling.<n>We argue that conformal calibration should be adopted as a standard component of machine-learning pipelines in collider physics.
  arXiv  Detail & Related papers  (2025-12-18T20:31:25Z)
• On Equivariant Model Selection through the Lens of Uncertainty [49.137341292207]
  Equivariant models leverage prior knowledge on symmetries to improve predictive performance, but misspecified architectural constraints can harm it instead.<n>We compare frequentist (via Conformal Prediction), Bayesian (via the marginal likelihood), and calibration-based measures to naive error-based evaluation.<n>We find that uncertainty metrics generally align with predictive performance, but Bayesian model evidence does so inconsistently.
  arXiv  Detail & Related papers  (2025-06-23T13:35:06Z)
• Solving Inverse Problems with FLAIR [59.02385492199431]
  Flow-based latent generative models are able to generate images with remarkable quality, even enabling text-to-image generation.<n>We present FLAIR, a novel training free variational framework that leverages flow-based generative models as a prior for inverse problems.<n>Results on standard imaging benchmarks demonstrate that FLAIR consistently outperforms existing diffusion- and flow-based methods in terms of reconstruction quality and sample diversity.
  arXiv  Detail & Related papers  (2025-06-03T09:29:47Z)
• CLUE: Neural Networks Calibration via Learning Uncertainty-Error alignment [7.702016079410588]
  We introduce CLUE (Calibration via Learning Uncertainty-Error Alignment), a novel approach that aligns predicted uncertainty with observed error during training.<n>We show that CLUE achieves superior calibration quality and competitive predictive performance with respect to state-of-the-art approaches.
  arXiv  Detail & Related papers  (2025-05-28T19:23:47Z)
• Beyond One-Hot Labels: Semantic Mixing for Model Calibration [22.39558434131574]
  We introduce calibration-aware data augmentation to create synthetic datasets of diverse samples and their ground-truth uncertainty. We propose calibrated reannotation to tackle the misalignment between the annotated confidence score and the mixing ratio. Experimental results demonstrate that CSM achieves superior calibration compared to the state-of-the-art calibration approaches.
  arXiv  Detail & Related papers  (2025-04-18T08:26:18Z)
• Embedded Nonlocal Operator Regression (ENOR): Quantifying model error in learning nonlocal operators [8.585650361148558]
  We propose a new framework to learn a nonlocal homogenized surrogate model and its structural model error. This framework provides discrepancy-adaptive uncertainty quantification for homogenized material response predictions in long-term simulations.
  arXiv  Detail & Related papers  (2024-10-27T04:17:27Z)
• Gradient-free variational learning with conditional mixture networks [39.827869318925494]
  We introduce CAVI-CMN, a fast, gradient-free variational method for training conditional mixture networks (CMNs)<n>CAVI-CMN achieves competitive and often superior predictive accuracy compared to maximum likelihood estimation (MLE) with backpropagation.<n>As input size or the number of experts increases, computation time scales competitively with MLE.
  arXiv  Detail & Related papers  (2024-08-29T10:43:55Z)
• Probabilistic Scores of Classifiers, Calibration is not Enough [0.32985979395737786]
  In binary classification tasks, accurate representation of probabilistic predictions is essential for various real-world applications. In this study, we highlight approaches that prioritize the alignment between predicted scores and true probability distributions. Our findings reveal limitations in traditional calibration metrics, which could undermine the reliability of predictive models for critical decision-making.
  arXiv  Detail & Related papers  (2024-08-06T19:53:00Z)
• Calibrating Large Language Models with Sample Consistency [76.23956851098598]
  We explore the potential of deriving confidence from the distribution of multiple randomly sampled model generations, via three measures of consistency. Results show that consistency-based calibration methods outperform existing post-hoc approaches. We offer practical guidance on choosing suitable consistency metrics for calibration, tailored to the characteristics of various LMs.
  arXiv  Detail & Related papers  (2024-02-21T16:15:20Z)
• Towards Understanding Variants of Invariant Risk Minimization through the Lens of Calibration [0.6906005491572401]
  We show that Information Bottleneck-based IRM achieves consistent calibration across different environments. Our empirical evidence indicates that models exhibiting consistent calibration across environments are also well-calibrated.
  arXiv  Detail & Related papers  (2024-01-31T02:08:43Z)
• Online Variational Sequential Monte Carlo [49.97673761305336]
  We build upon the variational sequential Monte Carlo (VSMC) method, which provides computationally efficient and accurate model parameter estimation and Bayesian latent-state inference. Online VSMC is capable of performing efficiently, entirely on-the-fly, both parameter estimation and particle proposal adaptation.
  arXiv  Detail & Related papers  (2023-12-19T21:45:38Z)
• Model Merging by Uncertainty-Based Gradient Matching [70.54580972266096]
  We propose a new uncertainty-based scheme to improve the performance by reducing the mismatch. Our new method gives consistent improvements for large language models and vision transformers.
  arXiv  Detail & Related papers  (2023-10-19T15:02:45Z)
• CLIPood: Generalizing CLIP to Out-of-Distributions [73.86353105017076]
  Contrastive language-image pre-training (CLIP) models have shown impressive zero-shot ability, but the further adaptation of CLIP on downstream tasks undesirably degrades OOD performances. We propose CLIPood, a fine-tuning method that can adapt CLIP models to OOD situations where both domain shifts and open classes may occur on unseen test data. Experiments on diverse datasets with different OOD scenarios show that CLIPood consistently outperforms existing generalization techniques.
  arXiv  Detail & Related papers  (2023-02-02T04:27:54Z)
• Modular Conformal Calibration [80.33410096908872]
  We introduce a versatile class of algorithms for recalibration in regression. This framework allows one to transform any regression model into a calibrated probabilistic model. We conduct an empirical study of MCC on 17 regression datasets.
  arXiv  Detail & Related papers  (2022-06-23T03:25:23Z)
• Autoregressive Score Matching [113.4502004812927]
  We propose autoregressive conditional score models (AR-CSM) where we parameterize the joint distribution in terms of the derivatives of univariable log-conditionals (scores) For AR-CSM models, this divergence between data and model distributions can be computed and optimized efficiently, requiring no expensive sampling or adversarial training. We show with extensive experimental results that it can be applied to density estimation on synthetic data, image generation, image denoising, and training latent variable models with implicit encoders.
  arXiv  Detail & Related papers  (2020-10-24T07:01:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.