A Practical Second-order Latent Factor Model via Distributed Particle Swarm Optimization

• URL: http://arxiv.org/abs/2208.06125v1
• Date: Fri, 12 Aug 2022 05:49:08 GMT
• Title: A Practical Second-order Latent Factor Model via Distributed Particle Swarm Optimization
• Authors: Jialiang Wang and Yurong Zhong and Weiling Li
• Abstract summary: Hessian-free (HF) optimization is an efficient method to utilizing second-order information of an LF model's objective function. A practical SLF (PSLF) model is proposed in this work. Experiments on real HiDS data sets indicate that PSLF model has a competitive advantage over state-of-the-art models in data representation ability.
• Score: 5.199454801210509
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Latent Factor (LF) models are effective in representing high-dimension and sparse (HiDS) data via low-rank matrices approximation. Hessian-free (HF) optimization is an efficient method to utilizing second-order information of an LF model's objective function and it has been utilized to optimize second-order LF (SLF) model. However, the low-rank representation ability of a SLF model heavily relies on its multiple hyperparameters. Determining these hyperparameters is time-consuming and it largely reduces the practicability of an SLF model. To address this issue, a practical SLF (PSLF) model is proposed in this work. It realizes hyperparameter self-adaptation with a distributed particle swarm optimizer (DPSO), which is gradient-free and parallelized. Experiments on real HiDS data sets indicate that PSLF model has a competitive advantage over state-of-the-art models in data representation ability.

Related papers

• Energy-Based Diffusion Language Models for Text Generation [126.23425882687195]
  Energy-based Diffusion Language Model (EDLM) is an energy-based model operating at the full sequence level for each diffusion step. Our framework offers a 1.3$times$ sampling speedup over existing diffusion models.
  arXiv  Detail & Related papers  (2024-10-28T17:25:56Z)
• PSLF: A PID Controller-incorporated Second-order Latent Factor Analysis Model for Recommender System [11.650076383080526]
  A second-order-based HDI model (SLF) analysis demonstrates superior performance in graph learning, particularly for high- and incomplete factor data rates.
  arXiv  Detail & Related papers  (2024-08-31T13:01:58Z)
• Spectrum-Aware Parameter Efficient Fine-Tuning for Diffusion Models [73.88009808326387]
  We propose a novel spectrum-aware adaptation framework for generative models. Our method adjusts both singular values and their basis vectors of pretrained weights. We introduce Spectral Ortho Decomposition Adaptation (SODA), which balances computational efficiency and representation capacity.
  arXiv  Detail & Related papers  (2024-05-31T17:43:35Z)
• Latent Semantic Consensus For Deterministic Geometric Model Fitting [109.44565542031384]
  We propose an effective method called Latent Semantic Consensus (LSC) LSC formulates the model fitting problem into two latent semantic spaces based on data points and model hypotheses. LSC is able to provide consistent and reliable solutions within only a few milliseconds for general multi-structural model fitting.
  arXiv  Detail & Related papers  (2024-03-11T05:35:38Z)
• Generative Modeling with Phase Stochastic Bridges [49.4474628881673]
  Diffusion models (DMs) represent state-of-the-art generative models for continuous inputs. We introduce a novel generative modeling framework grounded in textbfphase space dynamics Our framework demonstrates the capability to generate realistic data points at an early stage of dynamics propagation.
  arXiv  Detail & Related papers  (2023-10-11T18:38:28Z)
• Self-Supervised Dataset Distillation for Transfer Learning [77.4714995131992]
  We propose a novel problem of distilling an unlabeled dataset into a set of small synthetic samples for efficient self-supervised learning (SSL) We first prove that a gradient of synthetic samples with respect to a SSL objective in naive bilevel optimization is textitbiased due to randomness originating from data augmentations or masking. We empirically validate the effectiveness of our method on various applications involving transfer learning.
  arXiv  Detail & Related papers  (2023-10-10T10:48:52Z)
• Active-Learning-Driven Surrogate Modeling for Efficient Simulation of Parametric Nonlinear Systems [0.0]
  In absence of governing equations, we need to construct the parametric reduced-order surrogate model in a non-intrusive fashion. Our work provides a non-intrusive optimality criterion to efficiently populate the parameter snapshots. We propose an active-learning-driven surrogate model using kernel-based shallow neural networks.
  arXiv  Detail & Related papers  (2023-06-09T18:01:14Z)
• A Dynamic-Neighbor Particle Swarm Optimizer for Accurate Latent Factor Analysis [8.451827165005993]
  The performance of an LFA model heavily rely on its optimization process. Some prior studies employ the Particle Swarm Optimization to enhance an LFA model's optimization process. This paper proposes a Dynamic-neighbor-cooperated Hierarchical PSO-enhanced LFA model with two-fold main ideas.
  arXiv  Detail & Related papers  (2023-02-23T12:03:59Z)
• An Adaptive Alternating-direction-method-based Nonnegative Latent Factor Model [2.857044909410376]
  An alternating-direction-method-based nonnegative latent factor model can perform efficient representation learning to a high-dimensional and incomplete (HDI) matrix. This paper proposes an Adaptive Alternating-direction-method-based Nonnegative Latent Factor model, whose hyper- parameter adaptation is implemented following the principle of particle swarm optimization. Empirical studies on nonnegative HDI matrices generated by industrial applications indicate that A2NLF outperforms several state-of-the-art models in terms of computational and storage efficiency, as well as maintains highly competitive estimation accuracy for an HDI matrix's missing data
  arXiv  Detail & Related papers  (2022-04-11T03:04:26Z)
• A Data-driven feature selection and machine-learning model benchmark for the prediction of longitudinal dispersion coefficient [29.58577229101903]
  An accurate prediction on Longitudinal Dispersion(LD) coefficient can produce a performance leap in related simulation. In this study, a global optimal feature set was proposed through numerical comparison of the distilled local optimums in performance with representative ML models. Results show that the support vector machine has significantly better performance than other models.
  arXiv  Detail & Related papers  (2021-07-16T09:50:38Z)
• Autoregressive Score Matching [113.4502004812927]
  We propose autoregressive conditional score models (AR-CSM) where we parameterize the joint distribution in terms of the derivatives of univariable log-conditionals (scores) For AR-CSM models, this divergence between data and model distributions can be computed and optimized efficiently, requiring no expensive sampling or adversarial training. We show with extensive experimental results that it can be applied to density estimation on synthetic data, image generation, image denoising, and training latent variable models with implicit encoders.
  arXiv  Detail & Related papers  (2020-10-24T07:01:24Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.