Subsampling Factorization Machine Annealing

• URL: http://arxiv.org/abs/2508.08778v1
• Date: Tue, 12 Aug 2025 09:25:47 GMT
• Title: Subsampling Factorization Machine Annealing
• Authors: Yusuke Hama, Tadashi Kadowaki,
• Abstract summary: We develop an algorithm to solve a black-box optimization problem by improving Factorization Machine Annealing (FMA)<n>According to such a probabilistic training process, the performance of FMA on exploring a solution space gets enhanced.<n>We conduct numerical benchmarking tests to compare the performance of SFMA with that of FMA.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Quantum computing and machine learning are state-of-the-art technologies which have been investigated intensively in both academia and industry. The hybrid technology of these two ingredients is expected to be a powerful tool to solve complex problems in many branches of science and engineering such as combinatorial optimization problems and accelerate the creation of next-generation technologies. In this work, we develop an algorithm to solve a black-box optimization problem by improving Factorization Machine Annealing (FMA) such that the training of a machine learning model called Factorization Machine is performed not by a full dataset but by a subdataset which is sampled from a full dataset: Subsampling Factorization Machine Annealing (SFMA). According to such a probabilistic training process, the performance of FMA on exploring a solution space gets enhanced. As a result, SFMA exhibits balanced performance of exploration and exploitation which we call exploitation-exploration functionality. We conduct numerical benchmarking tests to compare the performance of SFMA with that of FMA. Consequently, SFMA certainly exhibits the exploration-exploitation functionality and outperforms FMA in speed and accuracy. In addition, the performance of SFMA can be further improved by sequentially using two subsampling datasets with different sizes such that the size of the latter dataset is substantially smaller than the former. Such a substantial reduction not only enhances the exploration performance of SFMA but also enables us to run it with correspondingly low computational cost even for a large-scale problem. These results indicate the effectiveness of SFMA in a certain class of black-box optimization problems of significant size: the potential scalability of SFMA in solving large-scale problems with correspondingly low computational cost.

Related papers

• Optimization Performance of Factorization Machine with Annealing under Limited Training Data [1.0937094979510213]
  Black-box (BB) optimization problems aim to identify an input that minimizes the output of a function whose input-output relationship is unknown.<n>Factorization machine (FM) is a surrogate model to iteratively guide the solution search via an Ising machine.<n>We propose a novel method for sequential dataset construction that retains at most a specified number of the most recently added data points.
  arXiv  Detail & Related papers  (2025-07-28T17:45:10Z)
• Reinforcement Learning-based Feature Generation Algorithm for Scientific Data [6.449769135199048]
  Feature generation (FG) aims to enhance the prediction potential of original data by constructing high-order feature combinations and removing redundant features.<n>This paper proposes the Multi-agent Feature Generation (MAFG) framework. Specifically, multi-agents will construct mathematical transformation equations collaboratively, synthesize and identify feature combinations ex-hibiting high information content, and leverage a reinforcement learning mechanism to evolve their strategies.
  arXiv  Detail & Related papers  (2025-07-04T11:52:09Z)
• DSMoE: Matrix-Partitioned Experts with Dynamic Routing for Computation-Efficient Dense LLMs [86.76714527437383]
  This paper proposes DSMoE, a novel approach that achieves sparsification by partitioning pre-trained FFN layers into computational blocks.<n>We implement adaptive expert routing using sigmoid activation and straight-through estimators, enabling tokens to flexibly access different aspects of model knowledge.<n>Experiments on LLaMA models demonstrate that under equivalent computational constraints, DSMoE achieves superior performance compared to existing pruning and MoE approaches.
  arXiv  Detail & Related papers  (2025-02-18T02:37:26Z)
• GSIFN: A Graph-Structured and Interlaced-Masked Multimodal Transformer-based Fusion Network for Multimodal Sentiment Analysis [0.0]
  Multimodal Sentiment Analysis (MSA) leverages multiple data modals to analyze human sentiment.<n>Existing MSA models generally employ cutting-edge multimodal fusion and representation learning-based methods to promote MSA capability.<n>Our proposed GSIFN incorporates two main components to solve these problems: (i) a graph-structured and interlaced-masked multimodal Transformer.<n>It adopts the Interlaced Mask mechanism to construct robust multimodal graph embedding, achieve all-modal-in-one Transformer-based fusion, and greatly reduce the computational overhead.
  arXiv  Detail & Related papers  (2024-08-27T06:44:28Z)
• FactorLLM: Factorizing Knowledge via Mixture of Experts for Large Language Models [50.331708897857574]
  We introduce FactorLLM, a novel approach that decomposes well-trained dense FFNs into sparse sub-networks without requiring any further modifications. FactorLLM achieves comparable performance to the source model securing up to 85% model performance while obtaining over a 30% increase in inference speed.
  arXiv  Detail & Related papers  (2024-08-15T16:45:16Z)
• Hybrid Heuristic Algorithms for Adiabatic Quantum Machine Learning Models [2.7407913606612615]
  This paper introduces a novel hybrid algorithm that incorporates an "r-flip" strategy.<n>This strategy is aimed at solving large-scale QUBO problems more effectively, offering better solution quality and lower computational costs.<n>The r-flip approach has practical applications in diverse fields, including cross-docking, supply chain management, machine scheduling, and fraud detection.
  arXiv  Detail & Related papers  (2024-07-26T19:31:58Z)
• Filling the Missing: Exploring Generative AI for Enhanced Federated Learning over Heterogeneous Mobile Edge Devices [72.61177465035031]
  We propose a generative AI-empowered federated learning to address these challenges by leveraging the idea of FIlling the MIssing (FIMI) portion of local data. Experiment results demonstrate that FIMI can save up to 50% of the device-side energy to achieve the target global test accuracy.
  arXiv  Detail & Related papers  (2023-10-21T12:07:04Z)
• Federated Conditional Stochastic Optimization [110.513884892319]
  Conditional optimization has found in a wide range of machine learning tasks, such as in-variant learning tasks, AUPRC, andAML. This paper proposes algorithms for distributed federated learning.
  arXiv  Detail & Related papers  (2023-10-04T01:47:37Z)
• Large Scale Mask Optimization Via Convolutional Fourier Neural Operator and Litho-Guided Self Training [54.16367467777526]
  We present a Convolutional Neural Operator (CFCF) that can efficiently learn mask tasks. For the first time, our machine learning-based framework outperforms state-of-the-art numerical mask dataset.
  arXiv  Detail & Related papers  (2022-07-08T16:39:31Z)
• Hyperparameter optimization of data-driven AI models on HPC systems [0.0]
  This work is part of RAISE's work on data-driven use cases which leverages AI- and HPC cross-methods. It is shown that in the case of Machine-Learned Particle reconstruction in High Energy Physics, the ASHA algorithm in combination with Bayesian optimization gives the largest performance increase per compute resources spent out of the investigated algorithms.
  arXiv  Detail & Related papers  (2022-03-02T14:02:59Z)
• Machine Learning Framework for Quantum Sampling of Highly-Constrained, Continuous Optimization Problems [101.18253437732933]
  We develop a generic, machine learning-based framework for mapping continuous-space inverse design problems into surrogate unconstrained binary optimization problems. We showcase the framework's performance on two inverse design problems by optimizing thermal emitter topologies for thermophotovoltaic applications and (ii) diffractive meta-gratings for highly efficient beam steering.
  arXiv  Detail & Related papers  (2021-05-06T02:22:23Z)
• Application of an automated machine learning-genetic algorithm (AutoML-GA) coupled with computational fluid dynamics simulations for rapid engine design optimization [0.0]
  The present work describes and validates an automated active learning approach, AutoML-GA, for surrogate-based optimization of internal combustion engines. A genetic algorithm is employed to locate the design optimum on the machine learning surrogate surface. It is demonstrated that AutoML-GA leads to a better optimum with a lower number of CFD simulations.
  arXiv  Detail & Related papers  (2021-01-07T17:50:52Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.