Adaptive Divergence-based Non-negative Latent Factor Analysis

• URL: http://arxiv.org/abs/2203.16214v1
• Date: Wed, 30 Mar 2022 11:28:36 GMT
• Title: Adaptive Divergence-based Non-negative Latent Factor Analysis
• Authors: Ye Yuan, Guangxiao Yuan, Renfang Wang, and Xin Luo
• Abstract summary: This study presents an Adaptive Divergence-based Non-negative Latent Factor (ADNLF) model with three-fold ideas. An ADNLF model achieves significantly higher estimation accuracy for missing data of an HDI dataset with high computational efficiency.
• Score: 6.265179945530255
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: High-Dimensional and Incomplete (HDI) data are frequently found in various industrial applications with complex interactions among numerous nodes, which are commonly non-negative for representing the inherent non-negativity of node interactions. A Non-negative Latent Factor (NLF) model is able to extract intrinsic features from such data efficiently. However, existing NLF models all adopt a static divergence metric like Euclidean distance or {\alpha}-\b{eta} divergence to build its learning objective, which greatly restricts its scalability of accurately representing HDI data from different domains. Aiming at addressing this issue, this study presents an Adaptive Divergence-based Non-negative Latent Factor (ADNLF) model with three-fold ideas: a) generalizing the objective function with the {\alpha}-\b{eta}-divergence to expand its potential of representing various HDI data; b) adopting a non-negative bridging function to connect the optimization variables with output latent factors for fulfilling the non-negativity constraints constantly; and c) making the divergence parameters adaptive through particle swarm optimization, thereby facilitating adaptive divergence in the learning objective to achieve high scalability. Empirical studies are conducted on four HDI datasets from real applications, whose results demonstrate that in comparison with state-of-the-art NLF models, an ADNLF model achieves significantly higher estimation accuracy for missing data of an HDI dataset with high computational efficiency.

Related papers

• Assumption-Lean Post-Integrated Inference with Negative Control Outcomes [0.0]
  We introduce a robust post-integrated inference (PII) method that adjusts for latent heterogeneity using negative control outcomes. Our assumption-lean semi inference method extends robustness and generality to projected direct effect estimands that account for mediators, confounders, and moderators. The proposed doubly robust estimators are consistent and efficient under minimal assumptions, facilitating data-adaptive estimation with machine learning algorithms.
  arXiv  Detail & Related papers  (2024-10-07T12:52:38Z)
• The Risk of Federated Learning to Skew Fine-Tuning Features and Underperform Out-of-Distribution Robustness [50.52507648690234]
  Federated learning has the risk of skewing fine-tuning features and compromising the robustness of the model. We introduce three robustness indicators and conduct experiments across diverse robust datasets. Our approach markedly enhances the robustness across diverse scenarios, encompassing various parameter-efficient fine-tuning methods.
  arXiv  Detail & Related papers  (2024-01-25T09:18:51Z)
• VTAE: Variational Transformer Autoencoder with Manifolds Learning [144.0546653941249]
  Deep generative models have demonstrated successful applications in learning non-linear data distributions through a number of latent variables. The nonlinearity of the generator implies that the latent space shows an unsatisfactory projection of the data space, which results in poor representation learning. We show that geodesics and accurate computation can substantially improve the performance of deep generative models.
  arXiv  Detail & Related papers  (2023-04-03T13:13:19Z)
• A Nonlinear PID-Enhanced Adaptive Latent Factor Analysis Model [6.2303427193075755]
  High-dimensional and incomplete (HDI) data holds tremendous interactive information in various industrial applications. A latent factor (LF) model is remarkably effective in extracting valuable information from HDI data with decent gradient (SGD) algorithm. An SGD-based LFA model suffers from slow convergence since it only considers the current learning error.
  arXiv  Detail & Related papers  (2022-08-04T07:48:19Z)
• PI-NLF: A Proportional-Integral Approach for Non-negative Latent Factor Analysis [9.087387628717952]
  A non-negative latent factor (NLF) model performs efficient representation learning to an HDI matrix. A PI-NLF model outperforms the state-of-the-art models in both computational efficiency and estimation accuracy for missing data of an HDI matrix.
  arXiv  Detail & Related papers  (2022-05-05T12:04:52Z)
• Scalable Regularised Joint Mixture Models [2.0686407686198263]
  In many applications, data can be heterogeneous in the sense of spanning latent groups with different underlying distributions. We propose an approach for heterogeneous data that allows joint learning of (i) explicit multivariate feature distributions, (ii) high-dimensional regression models and (iii) latent group labels. The approach is demonstrably effective in high dimensions, combining data reduction for computational efficiency with a re-weighting scheme that retains key signals even when the number of features is large.
  arXiv  Detail & Related papers  (2022-05-03T13:38:58Z)
• An Adaptive Alternating-direction-method-based Nonnegative Latent Factor Model [2.857044909410376]
  An alternating-direction-method-based nonnegative latent factor model can perform efficient representation learning to a high-dimensional and incomplete (HDI) matrix. This paper proposes an Adaptive Alternating-direction-method-based Nonnegative Latent Factor model, whose hyper- parameter adaptation is implemented following the principle of particle swarm optimization. Empirical studies on nonnegative HDI matrices generated by industrial applications indicate that A2NLF outperforms several state-of-the-art models in terms of computational and storage efficiency, as well as maintains highly competitive estimation accuracy for an HDI matrix's missing data
  arXiv  Detail & Related papers  (2022-04-11T03:04:26Z)
• Improving Generalization via Uncertainty Driven Perturbations [107.45752065285821]
  We consider uncertainty-driven perturbations of the training data points. Unlike loss-driven perturbations, uncertainty-guided perturbations do not cross the decision boundary. We show that UDP is guaranteed to achieve the robustness margin decision on linear models.
  arXiv  Detail & Related papers  (2022-02-11T16:22:08Z)
• Pseudo-Spherical Contrastive Divergence [119.28384561517292]
  We propose pseudo-spherical contrastive divergence (PS-CD) to generalize maximum learning likelihood of energy-based models. PS-CD avoids the intractable partition function and provides a generalized family of learning objectives.
  arXiv  Detail & Related papers  (2021-11-01T09:17:15Z)
• Probabilistic Circuits for Variational Inference in Discrete Graphical Models [101.28528515775842]
  Inference in discrete graphical models with variational methods is difficult. Many sampling-based methods have been proposed for estimating Evidence Lower Bound (ELBO) We propose a new approach that leverages the tractability of probabilistic circuit models, such as Sum Product Networks (SPN) We show that selective-SPNs are suitable as an expressive variational distribution, and prove that when the log-density of the target model is aweighted the corresponding ELBO can be computed analytically.
  arXiv  Detail & Related papers  (2020-10-22T05:04:38Z)
• Accounting for Unobserved Confounding in Domain Generalization [107.0464488046289]
  This paper investigates the problem of learning robust, generalizable prediction models from a combination of datasets. Part of the challenge of learning robust models lies in the influence of unobserved confounders. We demonstrate the empirical performance of our approach on healthcare data from different modalities.
  arXiv  Detail & Related papers  (2020-07-21T08:18:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.