Quantum Annealing based Feature Selection in Machine Learning

• URL: http://arxiv.org/abs/2411.19609v1
• Date: Fri, 29 Nov 2024 10:52:30 GMT
• Title: Quantum Annealing based Feature Selection in Machine Learning
• Authors: Daniel Pranjic, Bharadwaj Chowdary Mummaneni, Christian Tutschku,
• Abstract summary: Feature selection is crucial for enhancing the accuracy and efficiency of machine learning (ML) models. Calculating the optimal set of features that maximize the mutual information (MI) or conditional mutual information (CMI) is computationally intractable for large datasets on classical computers. This study employs a Mutual Information Quadratic Unconstrained Binary Optimization (MIQUBO) formulation, enabling its solution on a quantum annealer.
• Score: 0.6437284704257459
• License:
• Abstract: Feature selection is crucial for enhancing the accuracy and efficiency of machine learning (ML) models. This work investigates the utility of quantum annealing for the feature selection process in an ML-pipeline, used for maximizing the mutual information (MI) or conditional mutual information (CMI) of the underlying feature space. Calculating the optimal set of features that maximize the MI or CMI is computationally intractable for large datasets on classical computers, even with approximative methods. This study employs a Mutual Information Quadratic Unconstrained Binary Optimization (MIQUBO) formulation, enabling its solution on a quantum annealer. We demonstrate the capability of this approach to identify the best feature combinations that maximize the MI or CMI. To showcase its real-world applicability, we solve the MIQUBO problem to forecast the prices of used excavators. Our results demonstrate that for datasets with a small MI concentration the MIQUBO approach can provide a significant improvement over MI-only based approaches, dependent on the dimension of the selected feature space.

Related papers

• LLM-Select: Feature Selection with Large Language Models [64.5099482021597]
  Large language models (LLMs) are capable of selecting the most predictive features, with performance rivaling the standard tools of data science. Our findings suggest that LLMs may be useful not only for selecting the best features for training but also for deciding which features to collect in the first place.
  arXiv  Detail & Related papers  (2024-07-02T22:23:40Z)
• Machine Learning Optimized Approach for Parameter Selection in MESHFREE Simulations [0.0]
  Meshfree simulation methods are emerging as compelling alternatives to conventional mesh-based approaches. We provide a comprehensive overview of our research combining Machine Learning (ML) and Fraunhofer's MESHFREE software. We introduce a novel ML-optimized approach, using active learning, regression trees, and visualization on MESHFREE simulation data.
  arXiv  Detail & Related papers  (2024-03-20T15:29:59Z)
• AFS-BM: Enhancing Model Performance through Adaptive Feature Selection with Binary Masking [0.0]
  We introduce the "Adaptive Feature Selection with Binary Masking" (AFS-BM) We do the joint optimization and binary masking to continuously adapt the set of features and model parameters during the training process. Our results show that AFS-BM makes significant improvement in terms of accuracy and requires significantly less computational complexity.
  arXiv  Detail & Related papers  (2024-01-20T15:09:41Z)
• A novel feature selection method based on quantum support vector machine [3.6953740776904924]
  Feature selection is critical in machine learning to reduce dimensionality and improve model accuracy and efficiency. We propose a novel method, quantum support vector machine feature selection (QSVMF), integrating quantum support vector machines with genetic algorithm. We apply QSVMF for feature selection on a breast cancer dataset, comparing the performance of QSVMF against classical approaches.
  arXiv  Detail & Related papers  (2023-11-29T14:08:26Z)
• Self-Supervised Neuron Segmentation with Multi-Agent Reinforcement Learning [53.00683059396803]
  Mask image model (MIM) has been widely used due to its simplicity and effectiveness in recovering original information from masked images. We propose a decision-based MIM that utilizes reinforcement learning (RL) to automatically search for optimal image masking ratio and masking strategy. Our approach has a significant advantage over alternative self-supervised methods on the task of neuron segmentation.
  arXiv  Detail & Related papers  (2023-10-06T10:40:46Z)
• FAStEN: An Efficient Adaptive Method for Feature Selection and Estimation in High-Dimensional Functional Regressions [7.674715791336311]
  We propose a new, flexible and ultra-efficient approach to perform feature selection in a sparse function-on-function regression problem. We show how to extend it to the scalar-on-function framework. We present an application to brain fMRI data from the AOMIC PIOP1 study.
  arXiv  Detail & Related papers  (2023-03-26T19:41:17Z)
• Feature Selection for Classification with QAOA [11.516147824168732]
  Feature selection is of great importance in Machine Learning, where it can be used to reduce the dimensionality of classification, ranking and prediction problems. We consider in particular a quadratic feature selection problem that can be tackled with the Approximate Quantum Algorithm Optimization (QAOA), already employed in optimization. In our experiments, we consider seven different real-world datasets with dimensionality up to 21 and run QAOA on both a quantum simulator and, for small datasets, the 7-qubit IBM (ibm-perth) quantum computer.
  arXiv  Detail & Related papers  (2022-11-05T09:28:53Z)
• Learning with MISELBO: The Mixture Cookbook [62.75516608080322]
  We present the first ever mixture of variational approximations for a normalizing flow-based hierarchical variational autoencoder (VAE) with VampPrior and a PixelCNN decoder network. We explain this cooperative behavior by drawing a novel connection between VI and adaptive importance sampling. We obtain state-of-the-art results among VAE architectures in terms of negative log-likelihood on the MNIST and FashionMNIST datasets.
  arXiv  Detail & Related papers  (2022-09-30T15:01:35Z)
• Offline Model-Based Optimization via Normalized Maximum Likelihood Estimation [101.22379613810881]
  We consider data-driven optimization problems where one must maximize a function given only queries at a fixed set of points. This problem setting emerges in many domains where function evaluation is a complex and expensive process. We propose a tractable approximation that allows us to scale our method to high-capacity neural network models.
  arXiv  Detail & Related papers  (2021-02-16T06:04:27Z)
• Dif-MAML: Decentralized Multi-Agent Meta-Learning [54.39661018886268]
  We propose a cooperative multi-agent meta-learning algorithm, referred to as MAML or Dif-MAML. We show that the proposed strategy allows a collection of agents to attain agreement at a linear rate and to converge to a stationary point of the aggregate MAML. Simulation results illustrate the theoretical findings and the superior performance relative to the traditional non-cooperative setting.
  arXiv  Detail & Related papers  (2020-10-06T16:51:09Z)
• A Trainable Optimal Transport Embedding for Feature Aggregation and its Relationship to Attention [96.77554122595578]
  We introduce a parametrized representation of fixed size, which embeds and then aggregates elements from a given input set according to the optimal transport plan between the set and a trainable reference. Our approach scales to large datasets and allows end-to-end training of the reference, while also providing a simple unsupervised learning mechanism with small computational cost.
  arXiv  Detail & Related papers  (2020-06-22T08:35:58Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.