tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs)

• URL: http://arxiv.org/abs/2402.02456v2
• Date: Sat, 1 Jun 2024 15:54:54 GMT
• Title: tnGPS: Discovering Unknown Tensor Network Structure Search Algorithms via Large Language Models (LLMs)
• Authors: Junhua Zeng, Chao Li, Zhun Sun, Qibin Zhao, Guoxu Zhou,
• Abstract summary: We study how to harness large language models to automatically discover new tensor network structure search (TN-SS) algorithms. By observing how human experts innovate in research, we propose an automatic algorithm discovery framework called tnGPS. The proposed framework is an elaborate prompting pipeline that instruct LLMs to generate new TN-SS algorithms through iterative refinement and enhancement.
• Score: 31.69308712027795
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Tensor networks are efficient for extremely high-dimensional representation, but their model selection, known as tensor network structure search (TN-SS), is a challenging problem. Although several works have targeted TN-SS, most existing algorithms are manually crafted heuristics with poor performance, suffering from the curse of dimensionality and local convergence. In this work, we jump out of the box, studying how to harness large language models (LLMs) to automatically discover new TN-SS algorithms, replacing the involvement of human experts. By observing how human experts innovate in research, we model their common workflow and propose an automatic algorithm discovery framework called tnGPS. The proposed framework is an elaborate prompting pipeline that instruct LLMs to generate new TN-SS algorithms through iterative refinement and enhancement. The experimental results demonstrate that the algorithms discovered by tnGPS exhibit superior performance in benchmarks compared to the current state-of-the-art methods.

Related papers

• AutoGCN -- Towards Generic Human Activity Recognition with Neural Architecture Search [0.16385815610837165]
  This paper introduces AutoGCN, a generic Neural Architecture Search (NAS) algorithm for Human Activity Recognition (HAR) using Graph Convolution Networks (GCNs) We conduct extensive experiments on two large-scale datasets focused on skeleton-based action recognition to assess the proposed algorithm's performance.
  arXiv  Detail & Related papers  (2024-02-02T11:07:27Z)
• Genetically Modified Wolf Optimization with Stochastic Gradient Descent for Optimising Deep Neural Networks [0.0]
  This research aims to analyze an alternative approach to optimizing neural network (NN) weights, with the use of population-based metaheuristic algorithms. A hybrid between Grey Wolf (GWO) and Genetic Modified Algorithms (GA) is explored, in conjunction with Gradient Descent (SGD) This algorithm allows for a combination between exploitation and exploration, whilst also tackling the issue of high-dimensionality.
  arXiv  Detail & Related papers  (2023-01-21T13:22:09Z)
• Improved Algorithms for Neural Active Learning [74.89097665112621]
  We improve the theoretical and empirical performance of neural-network(NN)-based active learning algorithms for the non-parametric streaming setting. We introduce two regret metrics by minimizing the population loss that are more suitable in active learning than the one used in state-of-the-art (SOTA) related work.
  arXiv  Detail & Related papers  (2022-10-02T05:03:38Z)
• Permutation Search of Tensor Network Structures via Local Sampling [27.155329364896144]
  In this paper, we consider a practical variant of TN-SS, dubbed TN permutation search (TN-PS) We propose a practically-efficient algorithm to resolve the problem of TN-PS. Numerical results demonstrate that the new algorithm can reduce the required model size of TNs in extensive benchmarks.
  arXiv  Detail & Related papers  (2022-06-14T05:12:49Z)
• Self Semi Supervised Neural Architecture Search for Semantic Segmentation [6.488575826304023]
  We propose a Neural Architecture Search strategy based on self supervision and semi-supervised learning for the task of semantic segmentation. Our approach builds an optimized neural network model for this task. Experiments on the Cityscapes and PASCAL VOC 2012 datasets demonstrate that the discovered neural network is more efficient than a state-of-the-art hand-crafted NN model.
  arXiv  Detail & Related papers  (2022-01-29T19:49:44Z)
• Robust lEarned Shrinkage-Thresholding (REST): Robust unrolling for sparse recover [87.28082715343896]
  We consider deep neural networks for solving inverse problems that are robust to forward model mis-specifications. We design a new robust deep neural network architecture by applying algorithm unfolding techniques to a robust version of the underlying recovery problem. The proposed REST network is shown to outperform state-of-the-art model-based and data-driven algorithms in both compressive sensing and radar imaging problems.
  arXiv  Detail & Related papers  (2021-10-20T06:15:45Z)
• A SAR speckle filter based on Residual Convolutional Neural Networks [68.8204255655161]
  This work aims to present a novel method for filtering the speckle noise from Sentinel-1 data by applying Deep Learning (DL) algorithms, based on Convolutional Neural Networks (CNNs) The obtained results, if compared with the state of the art, show a clear improvement in terms of Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM)
  arXiv  Detail & Related papers  (2021-04-19T14:43:07Z)
• Evolving Reinforcement Learning Algorithms [186.62294652057062]
  We propose a method for meta-learning reinforcement learning algorithms. The learned algorithms are domain-agnostic and can generalize to new environments not seen during training. We highlight two learned algorithms which obtain good generalization performance over other classical control tasks, gridworld type tasks, and Atari games.
  arXiv  Detail & Related papers  (2021-01-08T18:55:07Z)
• Iterative Algorithm Induced Deep-Unfolding Neural Networks: Precoding Design for Multiuser MIMO Systems [59.804810122136345]
  We propose a framework for deep-unfolding, where a general form of iterative algorithm induced deep-unfolding neural network (IAIDNN) is developed. An efficient IAIDNN based on the structure of the classic weighted minimum mean-square error (WMMSE) iterative algorithm is developed. We show that the proposed IAIDNN efficiently achieves the performance of the iterative WMMSE algorithm with reduced computational complexity.
  arXiv  Detail & Related papers  (2020-06-15T02:57:57Z)
• Rectified Linear Postsynaptic Potential Function for Backpropagation in Deep Spiking Neural Networks [55.0627904986664]
  Spiking Neural Networks (SNNs) usetemporal spike patterns to represent and transmit information, which is not only biologically realistic but also suitable for ultra-low-power event-driven neuromorphic implementation. This paper investigates the contribution of spike timing dynamics to information encoding, synaptic plasticity and decision making, providing a new perspective to design of future DeepSNNs and neuromorphic hardware systems.
  arXiv  Detail & Related papers  (2020-03-26T11:13:07Z)
• A Supervised Learning Algorithm for Multilayer Spiking Neural Networks Based on Temporal Coding Toward Energy-Efficient VLSI Processor Design [2.6872737601772956]
  Spiking neural networks (SNNs) are brain-inspired mathematical models with the ability to process information in the form of spikes. We propose a novel supervised learning algorithm for SNNs based on temporal coding.
  arXiv  Detail & Related papers  (2020-01-08T03:37:08Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.