Learning Monotonic Probabilities with a Generative Cost Model

• URL: http://arxiv.org/abs/2506.03542v1
• Date: Wed, 04 Jun 2025 03:54:26 GMT
• Title: Learning Monotonic Probabilities with a Generative Cost Model
• Authors: Yongxiang Tang, Yanhua Cheng, Xiaocheng Liu, Chenchen Jiao, Yanxiang Zeng, Ning Luo, Pengjia Yuan, Xialong Liu, Peng Jiang,
• Abstract summary: This paper shows that the issue of strict monotonicity can be viewed as a partial order between an observable revenue variable and a latent cost variable.<n>We introduce a generative network for the latent cost variable, and propose the Implicit Generative Cost Model (IGCM) to address the implicit monotonic problem.
• Score: 6.846444442087272
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: In many machine learning tasks, it is often necessary for the relationship between input and output variables to be monotonic, including both strictly monotonic and implicitly monotonic relationships. Traditional methods for maintaining monotonicity mainly rely on construction or regularization techniques, whereas this paper shows that the issue of strict monotonic probability can be viewed as a partial order between an observable revenue variable and a latent cost variable. This perspective enables us to reformulate the monotonicity challenge into modeling the latent cost variable. To tackle this, we introduce a generative network for the latent cost variable, termed the Generative Cost Model (GCM), which inherently addresses the strict monotonic problem, and propose the Implicit Generative Cost Model (IGCM) to address the implicit monotonic problem. We further validate our approach with a numerical simulation of quantile regression and conduct multiple experiments on public datasets, showing that our method significantly outperforms existing monotonic modeling techniques. The code for our experiments can be found at https://github.com/tyxaaron/GCM.

Related papers

• MonoKAN: Certified Monotonic Kolmogorov-Arnold Network [48.623199394622546]
  In certain applications, model predictions must align with expert-imposed requirements, sometimes exemplified by partial monotonicity constraints. We introduce a novel ANN architecture called MonoKAN, based on the KAN architecture and achieves certified partial monotonicity while enhancing interpretability. Our experiments demonstrate that MonoKAN not only enhances interpretability but also improves predictive performance across the majority of benchmarks, outperforming state-of-the-art monotonic approaches.
  arXiv  Detail & Related papers  (2024-09-17T11:10:59Z)
• Beyond DAGs: A Latent Partial Causal Model for Multimodal Learning [80.44084021062105]
  We propose a novel latent partial causal model for multimodal data, featuring two latent coupled variables, connected by an undirected edge, to represent the transfer of knowledge across modalities.<n>Under specific statistical assumptions, we establish an identifiability result, demonstrating that representations learned by multimodal contrastive learning correspond to the latent coupled variables up to a trivial transformation.<n>Experiments on a pre-trained CLIP model embodies disentangled representations, enabling few-shot learning and improving domain generalization across diverse real-world datasets.
  arXiv  Detail & Related papers  (2024-02-09T07:18:06Z)
• Learning multi-modal generative models with permutation-invariant encoders and tighter variational objectives [5.549794481031468]
  Devising deep latent variable models for multi-modal data has been a long-standing theme in machine learning research. In this work, we consider a variational objective that can tightly approximate the data log-likelihood. We develop more flexible aggregation schemes that avoid the inductive biases in PoE or MoE approaches.
  arXiv  Detail & Related papers  (2023-09-01T10:32:21Z)
• Learning Joint Latent Space EBM Prior Model for Multi-layer Generator [44.4434704520236]
  We study the fundamental problem of learning multi-layer generator models. We propose an energy-based model (EBM) on the joint latent space over all layers of latent variables. Our experiments demonstrate that the learned model can be expressive in generating high-quality images.
  arXiv  Detail & Related papers  (2023-06-10T00:27:37Z)
• How to address monotonicity for model risk management? [1.0878040851638]
  This paper studies transparent neural networks in the presence of three types of monotonicity: individual monotonicity, weak pairwise monotonicity, and strong pairwise monotonicity. As a means of achieving monotonicity while maintaining transparency, we propose the monotonic groves of neural additive models.
  arXiv  Detail & Related papers  (2023-04-28T04:21:02Z)
• A study of latent monotonic attention variants [65.73442960456013]
  End-to-end models reach state-of-the-art performance for speech recognition, but global soft attention is not monotonic. We present a mathematically clean solution to introduce monotonicity, by introducing a new latent variable. We show that our monotonic models perform as good as the global soft attention model.
  arXiv  Detail & Related papers  (2021-03-30T22:35:56Z)
• Certified Monotonic Neural Networks [15.537695725617576]
  We propose to certify the monotonicity of the general piece-wise linear neural networks by solving a mixed integer linear programming problem. Our approach does not require human-designed constraints on the weight space and also yields more accurate approximation.
  arXiv  Detail & Related papers  (2020-11-20T04:58:13Z)
• Generative Temporal Difference Learning for Infinite-Horizon Prediction [101.59882753763888]
  We introduce the $gamma$-model, a predictive model of environment dynamics with an infinite probabilistic horizon. We discuss how its training reflects an inescapable tradeoff between training-time and testing-time compounding errors.
  arXiv  Detail & Related papers  (2020-10-27T17:54:12Z)
• GANs with Variational Entropy Regularizers: Applications in Mitigating the Mode-Collapse Issue [95.23775347605923]
  Building on the success of deep learning, Generative Adversarial Networks (GANs) provide a modern approach to learn a probability distribution from observed samples. GANs often suffer from the mode collapse issue where the generator fails to capture all existing modes of the input distribution. We take an information-theoretic approach and maximize a variational lower bound on the entropy of the generated samples to increase their diversity.
  arXiv  Detail & Related papers  (2020-09-24T19:34:37Z)
• Counterexample-Guided Learning of Monotonic Neural Networks [32.73558242733049]
  We focus on monotonicity constraints, which are common and require that the function's output increases with increasing values of specific input features. We develop a counterexample-guided technique to provably enforce monotonicity constraints at prediction time. We also propose a technique to use monotonicity as an inductive bias for deep learning.
  arXiv  Detail & Related papers  (2020-06-16T01:04:26Z)
• Exact Hard Monotonic Attention for Character-Level Transduction [76.66797368985453]
  We show that neural sequence-to-sequence models that use non-monotonic soft attention often outperform popular monotonic models. We develop a hard attention sequence-to-sequence model that enforces strict monotonicity and learns a latent alignment jointly while learning to transduce.
  arXiv  Detail & Related papers  (2019-05-15T17:51:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.