Optimization Performance of Factorization Machine with Annealing under Limited Training Data

• URL: http://arxiv.org/abs/2507.21024v1
• Date: Mon, 28 Jul 2025 17:45:10 GMT
• Title: Optimization Performance of Factorization Machine with Annealing under Limited Training Data
• Authors: Mayumi Nakano, Yuya Seki, Shuta Kikuchi, Shu Tanaka,
• Abstract summary: Black-box (BB) optimization problems aim to identify an input that minimizes the output of a function whose input-output relationship is unknown.<n>Factorization machine (FM) is a surrogate model to iteratively guide the solution search via an Ising machine.<n>We propose a novel method for sequential dataset construction that retains at most a specified number of the most recently added data points.
• Score: 1.0937094979510213
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Black-box (BB) optimization problems aim to identify an input that minimizes the output of a function (the BB function) whose input-output relationship is unknown. Factorization machine with annealing (FMA) is a promising approach to this task, employing a factorization machine (FM) as a surrogate model to iteratively guide the solution search via an Ising machine. Although FMA has demonstrated strong optimization performance across various applications, its performance often stagnates as the number of optimization iterations increases. One contributing factor to this stagnation is the growing number of data points in the dataset used to train FM. It is hypothesized that as more data points are accumulated, the contribution of newly added data points becomes diluted within the entire dataset, thereby reducing their impact on improving the prediction accuracy of FM. To address this issue, we propose a novel method for sequential dataset construction that retains at most a specified number of the most recently added data points. This strategy is designed to enhance the influence of newly added data points on the surrogate model. Numerical experiments demonstrate that the proposed FMA achieves lower-cost solutions with fewer BB function evaluations compared to the conventional FMA.

Related papers

• Addressing Delayed Feedback in Conversion Rate Prediction via Influence Functions [23.97164200705282]
  We propose an Influence Function-empowered framework for Delayed Feedback Modeling (IF-DFM)<n>IF-DFM leverages influence functions to estimate how newly acquired and delayed conversion data impact model parameters.<n>Experiments on benchmark datasets demonstrate that IF-DFM consistently surpasses state-of-the-art methods.
  arXiv  Detail & Related papers  (2025-02-01T16:23:13Z)
• Knowledge graph enhanced retrieval-augmented generation for failure mode and effects analysis [1.8849131083278732]
  This paper contributes by presenting set-theoretic standardization and a schema for FMEA data, an algorithm for creating vector embeddings from the FMEA-KG, and a KG-enhanced RAG framework.<n>Our approach is validated through a user experience design study, and we measure the precision and performance of the context retrieval recall.
  arXiv  Detail & Related papers  (2024-06-26T07:02:49Z)
• A PSO Based Method to Generate Actionable Counterfactuals for High Dimensional Data [3.0320603363468845]
  We describe an efficient and an actionable counterfactual (CF) generation method based on particle swarm optimization (PSO) An algorithm is proposed that incorporates these features and it enables greater control over the proximity and sparsity properties over the generated CFs.
  arXiv  Detail & Related papers  (2023-09-30T18:08:00Z)
• Analysis and Optimization of Wireless Federated Learning with Data Heterogeneity [72.85248553787538]
  This paper focuses on performance analysis and optimization for wireless FL, considering data heterogeneity, combined with wireless resource allocation. We formulate the loss function minimization problem, under constraints on long-term energy consumption and latency, and jointly optimize client scheduling, resource allocation, and the number of local training epochs (CRE) Experiments on real-world datasets demonstrate that the proposed algorithm outperforms other benchmarks in terms of the learning accuracy and energy consumption.
  arXiv  Detail & Related papers  (2023-08-04T04:18:01Z)
• Scaling Relationship on Learning Mathematical Reasoning with Large Language Models [75.29595679428105]
  We investigate how the pre-training loss, supervised data amount, and augmented data amount influence the reasoning performances of a supervised LLM. We find that rejection samples from multiple models push LLaMA-7B to an accuracy of 49.3% on GSM8K which outperforms the supervised fine-tuning (SFT) accuracy of 35.9% significantly.
  arXiv  Detail & Related papers  (2023-08-03T15:34:01Z)
• A machine learning approach to the prediction of heat-transfer coefficients in micro-channels [4.724825031148412]
  The accurate prediction of the two-phase heat transfer coefficient (HTC) is key to the optimal design and operation of compact heat exchangers. We use a multi-output Gaussian process regression (GPR) to estimate the HTC in microchannels as a function of the mass flow rate, heat flux, system pressure and channel diameter and length.
  arXiv  Detail & Related papers  (2023-05-28T15:48:01Z)
• To Repeat or Not To Repeat: Insights from Scaling LLM under Token-Crisis [50.31589712761807]
  Large language models (LLMs) are notoriously token-hungry during pre-training, and high-quality text data on the web is approaching its scaling limit for LLMs. We investigate the consequences of repeating pre-training data, revealing that the model is susceptible to overfitting. Second, we examine the key factors contributing to multi-epoch degradation, finding that significant factors include dataset size, model parameters, and training objectives.
  arXiv  Detail & Related papers  (2023-05-22T17:02:15Z)
• Personalized Federated Learning under Mixture of Distributions [98.25444470990107]
  We propose a novel approach to Personalized Federated Learning (PFL), which utilizes Gaussian mixture models (GMM) to fit the input data distributions across diverse clients. FedGMM possesses an additional advantage of adapting to new clients with minimal overhead, and it also enables uncertainty quantification. Empirical evaluations on synthetic and benchmark datasets demonstrate the superior performance of our method in both PFL classification and novel sample detection.
  arXiv  Detail & Related papers  (2023-05-01T20:04:46Z)
• FAStEN: An Efficient Adaptive Method for Feature Selection and Estimation in High-Dimensional Functional Regressions [7.674715791336311]
  We propose a new, flexible and ultra-efficient approach to perform feature selection in a sparse function-on-function regression problem. We show how to extend it to the scalar-on-function framework. We present an application to brain fMRI data from the AOMIC PIOP1 study.
  arXiv  Detail & Related papers  (2023-03-26T19:41:17Z)
• Boosting Factorization Machines via Saliency-Guided Mixup [125.15872106335692]
  We present MixFM, inspired by Mixup, to generate auxiliary training data to boost Factorization machines (FMs) We also put forward a novel Factorization Machine powered by Saliency-guided Mixup (denoted as SMFM)
  arXiv  Detail & Related papers  (2022-06-17T09:49:00Z)
• Rectified Max-Value Entropy Search for Bayesian Optimization [54.26984662139516]
  We develop a rectified MES acquisition function based on the notion of mutual information. As a result, RMES shows a consistent improvement over MES in several synthetic function benchmarks and real-world optimization problems.
  arXiv  Detail & Related papers  (2022-02-28T08:11:02Z)
• Memory-Efficient Factorization Machines via Binarizing both Data and Model Coefficients [9.692334398809457]
  Subspace imating machine (SEFM) has been proposed to overcome the limitation of Factorization Machines (FM) We propose a new method called Binarized FM which constraints the model parameters to be binary values. Our proposed method achieves comparable accuracy with SEFM but with much less memory cost.
  arXiv  Detail & Related papers  (2021-08-17T03:30:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.