Label Distribution Learning using the Squared Neural Family on the Probability Simplex

• URL: http://arxiv.org/abs/2412.07324v1
• Date: Tue, 10 Dec 2024 09:12:02 GMT
• Title: Label Distribution Learning using the Squared Neural Family on the Probability Simplex
• Authors: Daokun Zhang, Russell Tsuchida, Dino Sejdinovic,
• Abstract summary: We estimate a probability distribution of all possible label distributions over the simplex.<n>With the modeled distribution, label distribution prediction can be achieved by performing the expectation operation.<n>More information about the label distribution can be inferred, such as the prediction reliability and uncertainties.
• Score: 15.680835401104247
• License: http://creativecommons.org/licenses/by-nc-sa/4.0/
• Abstract: Label distribution learning (LDL) provides a framework wherein a distribution over categories rather than a single category is predicted, with the aim of addressing ambiguity in labeled data. Existing research on LDL mainly focuses on the task of point estimation, i.e., pinpointing an optimal distribution in the probability simplex conditioned on the input sample. In this paper, we estimate a probability distribution of all possible label distributions over the simplex, by unleashing the expressive power of the recently introduced Squared Neural Family (SNEFY). With the modeled distribution, label distribution prediction can be achieved by performing the expectation operation to estimate the mean of the distribution of label distributions. Moreover, more information about the label distribution can be inferred, such as the prediction reliability and uncertainties. We conduct extensive experiments on the label distribution prediction task, showing that our distribution modeling based method can achieve very competitive label distribution prediction performance compared with the state-of-the-art baselines. Additional experiments on active learning and ensemble learning demonstrate that our probabilistic approach can effectively boost the performance in these settings, by accurately estimating the prediction reliability and uncertainties.

Related papers

• Concentration Distribution Learning from Label Distributions [19.650545340162726]
  We come up with a new concept named background concentration to serve as the absolute description degree term of the label distribution.<n>We propose a novel model by probabilistic methods and neural networks to learn label distributions and background concentrations from existing LDL datasets.
  arXiv  Detail & Related papers  (2025-05-27T07:53:27Z)
• Label Distribution Learning with Biased Annotations by Learning Multi-Label Representation [120.97262070068224]
  Multi-label learning (MLL) has gained attention for its ability to represent real-world data.<n>Label Distribution Learning (LDL) faces challenges in collecting accurate label distributions.
  arXiv  Detail & Related papers  (2025-02-03T09:04:03Z)
• Continuous Contrastive Learning for Long-Tailed Semi-Supervised Recognition [50.61991746981703]
  Current state-of-the-art LTSSL approaches rely on high-quality pseudo-labels for large-scale unlabeled data. This paper introduces a novel probabilistic framework that unifies various recent proposals in long-tail learning. We introduce a continuous contrastive learning method, CCL, extending our framework to unlabeled data using reliable and smoothed pseudo-labels.
  arXiv  Detail & Related papers  (2024-10-08T15:06:10Z)
• Self-Knowledge Distillation for Learning Ambiguity [11.755814660833549]
  Recent language models often over-confidently predict a single label without consideration for its correctness. We propose a novel self-knowledge distillation method that enables models to learn label distributions more accurately. We validate our method on diverse NLU benchmark datasets and the experimental results demonstrate its effectiveness in producing better label distributions.
  arXiv  Detail & Related papers  (2024-06-14T05:11:32Z)
• SimPro: A Simple Probabilistic Framework Towards Realistic Long-Tailed Semi-Supervised Learning [49.94607673097326]
  We propose a highly adaptable framework, designated as SimPro, which does not rely on any predefined assumptions about the distribution of unlabeled data. Our framework, grounded in a probabilistic model, innovatively refines the expectation-maximization algorithm. Our method showcases consistent state-of-the-art performance across diverse benchmarks and data distribution scenarios.
  arXiv  Detail & Related papers  (2024-02-21T03:39:04Z)
• Uncertainty Quantification via Stable Distribution Propagation [60.065272548502]
  We propose a new approach for propagating stable probability distributions through neural networks. Our method is based on local linearization, which we show to be an optimal approximation in terms of total variation distance for the ReLU non-linearity.
  arXiv  Detail & Related papers  (2024-02-13T09:40:19Z)
• Dr. FERMI: A Stochastic Distributionally Robust Fair Empirical Risk Minimization Framework [12.734559823650887]
  In the presence of distribution shifts, fair machine learning models may behave unfairly on test data. Existing algorithms require full access to data and cannot be used when small batches are used. This paper proposes the first distributionally robust fairness framework with convergence guarantees that do not require knowledge of the causal graph.
  arXiv  Detail & Related papers  (2023-09-20T23:25:28Z)
• Distribution Shift Inversion for Out-of-Distribution Prediction [57.22301285120695]
  We propose a portable Distribution Shift Inversion algorithm for Out-of-Distribution (OoD) prediction. We show that our method provides a general performance gain when plugged into a wide range of commonly used OoD algorithms.
  arXiv  Detail & Related papers  (2023-06-14T08:00:49Z)
• Statistical Inference Under Constrained Selection Bias [20.862583584531322]
  We propose a framework that enables statistical inference in the presence of selection bias. The output is high-probability bounds on the value of an estimand for the target distribution. We analyze the computational and statistical properties of methods to estimate these bounds and show that our method can produce informative bounds on a variety of simulated and semisynthetic tasks.
  arXiv  Detail & Related papers  (2023-06-05T23:05:26Z)
• Label Distribution Learning from Logical Label [19.632157794117553]
  Label distribution learning (LDL) is an effective method to predict the label description degree (a.k.a. label distribution) of a sample. But annotating label distribution for training samples is extremely costly. We propose a novel method to learn an LDL model directly from the logical label, which unifies LE and LDL into a joint model.
  arXiv  Detail & Related papers  (2023-03-13T04:31:35Z)
• Dist-PU: Positive-Unlabeled Learning from a Label Distribution Perspective [89.5370481649529]
  We propose a label distribution perspective for PU learning in this paper. Motivated by this, we propose to pursue the label distribution consistency between predicted and ground-truth label distributions. Experiments on three benchmark datasets validate the effectiveness of the proposed method.
  arXiv  Detail & Related papers  (2022-12-06T07:38:29Z)
• Label distribution learning via label correlation grid [9.340734188957727]
  We propose a textbfLabel textbfCorrelation textbfGrid (LCG) to model the uncertainty of label relationships. Our network learns the LCG to accurately estimate the label distribution for each instance.
  arXiv  Detail & Related papers  (2022-10-15T03:58:15Z)
• Gray Learning from Non-IID Data with Out-of-distribution Samples [45.788789553551176]
  The integrity of training data, even when annotated by experts, is far from guaranteed. We introduce a novel approach, termed textitGray Learning, which leverages both ground-truth and complementary labels. By grounding our approach in statistical learning theory, we derive bounds for the generalization error, demonstrating that GL achieves tight constraints even in non-IID settings.
  arXiv  Detail & Related papers  (2022-06-19T10:46:38Z)
• Unimodal-Concentrated Loss: Fully Adaptive Label Distribution Learning for Ordinal Regression [32.35098925000738]
  We argue that existing ALDL algorithms do not fully exploit the intrinsic properties of ordinal regression. We propose a novel loss function for fully adaptive label distribution learning, namely unimodal-concentrated loss.
  arXiv  Detail & Related papers  (2022-04-01T09:40:11Z)
• Instance-Dependent Partial Label Learning [69.49681837908511]
  Partial label learning is a typical weakly supervised learning problem. Most existing approaches assume that the incorrect labels in each training example are randomly picked as the candidate labels. In this paper, we consider instance-dependent and assume that each example is associated with a latent label distribution constituted by the real number of each label.
  arXiv  Detail & Related papers  (2021-10-25T12:50:26Z)
• Personalized Trajectory Prediction via Distribution Discrimination [78.69458579657189]
  Trarimiy prediction is confronted with the dilemma to capture the multi-modal nature of future dynamics. We present a distribution discrimination (DisDis) method to predict personalized motion patterns. Our method can be integrated with existing multi-modal predictive models as a plug-and-play module.
  arXiv  Detail & Related papers  (2021-07-29T17:42:12Z)
• Mandoline: Model Evaluation under Distribution Shift [8.007644303175395]
  Machine learning models are often deployed in different settings than they were trained and validated on. We develop Mandoline, a new evaluation framework that mitigates these issues. Users write simple "slicing functions" - noisy, potentially correlated binary functions intended to capture possible axes of distribution shift.
  arXiv  Detail & Related papers  (2021-07-01T17:57:57Z)
• Disentangling Label Distribution for Long-tailed Visual Recognition [8.538887358164438]
  We formulate long-tailed visual recognition as a label shift problem where the target and source label distributions are different. One of the significant hurdles in dealing with the label shift problem is the entanglement between the source label distribution and the model prediction. We propose a novel method, LAbel distribution DisEntangling (LADE) loss based on the optimal bound of Donsker-Varadhan representation.
  arXiv  Detail & Related papers  (2020-12-01T07:56:53Z)
• Fair Densities via Boosting the Sufficient Statistics of Exponential Families [72.34223801798422]
  We introduce a boosting algorithm to pre-process data for fairness. Our approach shifts towards better data fitting while still ensuring a minimal fairness guarantee. Empirical results are present to display the quality of result on real-world data.
  arXiv  Detail & Related papers  (2020-12-01T00:49:17Z)
• Distributional Reinforcement Learning via Moment Matching [54.16108052278444]
  We formulate a method that learns a finite set of statistics from each return distribution via neural networks. Our method can be interpreted as implicitly matching all orders of moments between a return distribution and its Bellman target. Experiments on the suite of Atari games show that our method outperforms the standard distributional RL baselines.
  arXiv  Detail & Related papers  (2020-07-24T05:18:17Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.