$FM^2$: Field-matrixed Factorization Machines for Recommender Systems

• URL: http://arxiv.org/abs/2102.12994v2
• Date: Fri, 19 Mar 2021 17:19:47 GMT
• Title: $FM^2$: Field-matrixed Factorization Machines for Recommender Systems
• Authors: Yang Sun, Junwei Pan, Alex Zhang, Aaron Flores
• Abstract summary: We propose a novel approach to model the field information effectively and efficiently. The proposed approach is a direct improvement of FwFM, and is named as Field-matrixed Factorization Machines (FmFM)
• Score: 9.461169933697379
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Click-through rate (CTR) prediction plays a critical role in recommender systems and online advertising. The data used in these applications are multi-field categorical data, where each feature belongs to one field. Field information is proved to be important and there are several works considering fields in their models. In this paper, we proposed a novel approach to model the field information effectively and efficiently. The proposed approach is a direct improvement of FwFM, and is named as Field-matrixed Factorization Machines (FmFM, or $FM^2$). We also proposed a new explanation of FM and FwFM within the FmFM framework, and compared it with the FFM. Besides pruning the cross terms, our model supports field-specific variable dimensions of embedding vectors, which acts as soft pruning. We also proposed an efficient way to minimize the dimension while keeping the model performance. The FmFM model can also be optimized further by caching the intermediate vectors, and it only takes thousands of floating-point operations (FLOPs) to make a prediction. Our experiment results show that it can out-perform the FFM, which is more complex. The FmFM model's performance is also comparable to DNN models which require much more FLOPs in runtime.

Related papers

• Fed-piLot: Optimizing LoRA Assignment for Efficient Federated Foundation Model Fine-Tuning [11.10244162253018]
  We introduce Fed-piLot, an efficient FedFM fine-tuning framework with optimized local LoRA assignments for heterogeneous clients. We design a Local-Global Information Gain Score (IG-Score) based value function to optimize LoRA assignment under clients' memory constraints. Experimental results on three datasets under both IID and non-IID conditions demonstrate the effectiveness and efficiency of Fed-piLot.
  arXiv  Detail & Related papers  (2024-10-14T06:36:41Z)
• DFM: Interpolant-free Dual Flow Matching [0.8192907805418583]
  We propose an interpolant-free dual flow matching (DFM) approach without explicit assumptions about the modeled vector field. Experiments with the SMAP unsupervised anomaly detection show advantages of DFM when compared to the CNF trained with either maximum likelihood or FM objectives.
  arXiv  Detail & Related papers  (2024-10-11T20:46:04Z)
• FedPFT: Federated Proxy Fine-Tuning of Foundation Models [55.58899993272904]
  Adapting Foundation Models (FMs) for downstream tasks through Federated Learning (FL) emerges as a promising strategy for protecting data privacy and valuable FMs. Existing methods fine-tune FM by allocating sub-FM to clients in FL, leading to suboptimal performance due to insufficient tuning and inevitable error accumulations of gradients. We propose Federated Proxy Fine-Tuning (FedPFT), a novel method enhancing FMs adaptation in downstream tasks through FL by two key modules.
  arXiv  Detail & Related papers  (2024-04-17T16:30:06Z)
• Learn From Model Beyond Fine-Tuning: A Survey [78.80920533793595]
  Learn From Model (LFM) focuses on the research, modification, and design of foundation models (FM) based on the model interface. The study of LFM techniques can be broadly categorized into five major areas: model tuning, model distillation, model reuse, meta learning and model editing. This paper gives a comprehensive review of the current methods based on FM from the perspective of LFM.
  arXiv  Detail & Related papers  (2023-10-12T10:20:36Z)
• EMOFM: Ensemble MLP mOdel with Feature-based Mixers for Click-Through Rate Prediction [5.983194751474721]
  A dataset contains millions of records and each field-wise feature in a record consists of hashed integers for privacy. For this task, the keys of network-based methods might be type-wise feature extraction and information fusion across different fields. We propose plug-in mixers for field/type-wise feature fusion, thus construct an field&type-wise ensemble model, namely EMOFM.
  arXiv  Detail & Related papers  (2023-10-06T12:32:23Z)
• Improving and generalizing flow-based generative models with minibatch optimal transport [90.01613198337833]
  We introduce the generalized conditional flow matching (CFM) technique for continuous normalizing flows (CNFs) CFM features a stable regression objective like that used to train the flow in diffusion models but enjoys the efficient inference of deterministic flow models. A variant of our objective is optimal transport CFM (OT-CFM), which creates simpler flows that are more stable to train and lead to faster inference.
  arXiv  Detail & Related papers  (2023-02-01T14:47:17Z)
• Boosting Factorization Machines via Saliency-Guided Mixup [125.15872106335692]
  We present MixFM, inspired by Mixup, to generate auxiliary training data to boost Factorization machines (FMs) We also put forward a novel Factorization Machine powered by Saliency-guided Mixup (denoted as SMFM)
  arXiv  Detail & Related papers  (2022-06-17T09:49:00Z)
• Leaf-FM: A Learnable Feature Generation Factorization Machine for Click-Through Rate Prediction [2.412497918389292]
  We propose LeafFM model based on FM to generate new features from the original feature embedding by learning the transformation functions automatically. Experiments are conducted on three real-world datasets and the results show Leaf-FM model outperforms standard FMs by a large margin.
  arXiv  Detail & Related papers  (2021-07-26T08:29:18Z)
• Mixed Variable Bayesian Optimization with Frequency Modulated Kernels [96.78099706164747]
  We propose the frequency modulated (FM) kernel flexibly modeling dependencies among different types of variables. BO-FM outperforms competitors including Regularized evolution(RE) and BOHB.
  arXiv  Detail & Related papers  (2021-02-25T11:28:46Z)
• Learning Discrete Energy-based Models via Auxiliary-variable Local Exploration [130.89746032163106]
  We propose ALOE, a new algorithm for learning conditional and unconditional EBMs for discrete structured data. We show that the energy function and sampler can be trained efficiently via a new variational form of power iteration. We present an energy model guided fuzzer for software testing that achieves comparable performance to well engineered fuzzing engines like libfuzzer.
  arXiv  Detail & Related papers  (2020-11-10T19:31:29Z)
• Field-Embedded Factorization Machines for Click-through rate prediction [2.942829992746068]
  Click-through rate (CTR) prediction models are common in many online applications such as digital advertising and recommender systems. We propose a novel shallow Field-Embedded Factorization Machine (FEFM) and its deep counterpart Deep Field-Embedded Factorization Machine (DeepFEFM) FEFM has significantly lower model complexity than FFM and roughly the same complexity as FwFM.
  arXiv  Detail & Related papers  (2020-09-13T15:32:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.