Boosting Factorization Machines via Saliency-Guided Mixup
- URL: http://arxiv.org/abs/2206.08661v1
- Date: Fri, 17 Jun 2022 09:49:00 GMT
- Title: Boosting Factorization Machines via Saliency-Guided Mixup
- Authors: Chenwang Wu, Defu Lian, Yong Ge, Min Zhou, Enhong Chen, Dacheng Tao
- Abstract summary: We present MixFM, inspired by Mixup, to generate auxiliary training data to boost Factorization machines (FMs)
We also put forward a novel Factorization Machine powered by Saliency-guided Mixup (denoted as SMFM)
- Score: 125.15872106335692
- License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
- Abstract: Factorization machines (FMs) are widely used in recommender systems due to
their adaptability and ability to learn from sparse data. However, for the
ubiquitous non-interactive features in sparse data, existing FMs can only
estimate the parameters corresponding to these features via the inner product
of their embeddings. Undeniably, they cannot learn the direct interactions of
these features, which limits the model's expressive power. To this end, we
first present MixFM, inspired by Mixup, to generate auxiliary training data to
boost FMs. Unlike existing augmentation strategies that require labor costs and
expertise to collect additional information such as position and fields, these
extra data generated by MixFM only by the convex combination of the raw ones
without any professional knowledge support. More importantly, if the parent
samples to be mixed have non-interactive features, MixFM will establish their
direct interactions. Second, considering that MixFM may generate redundant or
even detrimental instances, we further put forward a novel Factorization
Machine powered by Saliency-guided Mixup (denoted as SMFM). Guided by the
customized saliency, SMFM can generate more informative neighbor data. Through
theoretical analysis, we prove that the proposed methods minimize the upper
bound of the generalization error, which hold a beneficial effect on enhancing
FMs. Significantly, we give the first generalization bound of FM, implying the
generalization requires more data and a smaller embedding size under the
sufficient representation capability. Finally, extensive experiments on five
datasets confirm that our approaches are superior to baselines. Besides, the
results show that "poisoning" mixed data is likewise beneficial to the FM
variants.
Related papers
- FedPFT: Federated Proxy Fine-Tuning of Foundation Models [55.58899993272904]
Adapting Foundation Models (FMs) for downstream tasks through Federated Learning (FL) emerges as a promising strategy for protecting data privacy and valuable FMs.
Existing methods fine-tune FM by allocating sub-FM to clients in FL, leading to suboptimal performance due to insufficient tuning and inevitable error accumulations of gradients.
We propose Federated Proxy Fine-Tuning (FedPFT), a novel method enhancing FMs adaptation in downstream tasks through FL by two key modules.
arXiv Detail & Related papers (2024-04-17T16:30:06Z) - Personalized Federated Learning under Mixture of Distributions [98.25444470990107]
We propose a novel approach to Personalized Federated Learning (PFL), which utilizes Gaussian mixture models (GMM) to fit the input data distributions across diverse clients.
FedGMM possesses an additional advantage of adapting to new clients with minimal overhead, and it also enables uncertainty quantification.
Empirical evaluations on synthetic and benchmark datasets demonstrate the superior performance of our method in both PFL classification and novel sample detection.
arXiv Detail & Related papers (2023-05-01T20:04:46Z) - FedFM: Anchor-based Feature Matching for Data Heterogeneity in Federated
Learning [91.74206675452888]
We propose a novel method FedFM, which guides each client's features to match shared category-wise anchors.
To achieve higher efficiency and flexibility, we propose a FedFM variant, called FedFM-Lite, where clients communicate with server with fewer synchronization times and communication bandwidth costs.
arXiv Detail & Related papers (2022-10-14T08:11:34Z) - Noisy Feature Mixup [42.056684988818766]
We introduce Noisy Feature Mixup (NFM), an inexpensive yet effective method for data augmentation.
NFM includes mixup and manifold mixup as special cases, but it has additional advantages, including better smoothing of decision boundaries.
We show that residual networks and vision transformers trained with NFM have favorable trade-offs between predictive accuracy on clean data and robustness with respect to various types of data.
arXiv Detail & Related papers (2021-10-05T17:13:51Z) - Quaternion Factorization Machines: A Lightweight Solution to Intricate
Feature Interaction Modelling [76.89779231460193]
factorization machine (FM) is capable of automatically learning high-order interactions among features to make predictions without the need for manual feature engineering.
We propose the quaternion factorization machine (QFM) and quaternion neural factorization machine (QNFM) for sparse predictive analytics.
arXiv Detail & Related papers (2021-04-05T00:02:36Z) - Suppressing Mislabeled Data via Grouping and Self-Attention [60.14212694011875]
Deep networks achieve excellent results on large-scale clean data but degrade significantly when learning from noisy labels.
This paper proposes a conceptually simple yet efficient training block, termed as Attentive Feature Mixup (AFM)
It allows paying more attention to clean samples and less to mislabeled ones via sample interactions in small groups.
arXiv Detail & Related papers (2020-10-29T13:54:16Z) - Factorization Machines with Regularization for Sparse Feature
Interactions [13.593781209611112]
Factorization machines (FMs) are machine learning predictive models based on second-order feature interactions.
We present a new regularization scheme for feature interaction selection in FMs.
For feature interaction selection, our proposed regularizer makes the feature interaction matrix sparse without a restriction on sparsity patterns imposed by the existing methods.
arXiv Detail & Related papers (2020-10-19T05:00:40Z) - Generalized Embedding Machines for Recommender Systems [10.8585932535286]
We propose an alternative approach to model high-order interaction signals in the embedding level, namely Generalized Embedding Machine (GEM)
In this paper, we utilize graph convolution networks (GCN) to generate high-order embeddings.
arXiv Detail & Related papers (2020-02-16T12:03:18Z)
This list is automatically generated from the titles and abstracts of the papers in this site.
This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.