Tensor Network for Supervised Learning at Finite Temperature

• URL: http://arxiv.org/abs/2104.05439v1
• Date: Fri, 9 Apr 2021 05:02:36 GMT
• Title: Tensor Network for Supervised Learning at Finite Temperature
• Authors: Haoxiang Lin, Shuqian Ye, Xi Zhu
• Abstract summary: finite temperature tensor network (FTTN) imports thermal perturbation into matrix product states framework. FTTN regards it as thermal loss computed from entanglement with environment. Temperature-like parameter can be automatically optimized, which gives each database an individual temperature.
• Score: 1.4717465036484292
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: The large variation of datasets is a huge barrier for image classification tasks. In this paper, we embraced this observation and introduce the finite temperature tensor network (FTTN), which imports the thermal perturbation into the matrix product states framework by placing all images in an environment with constant temperature, in analog to energy-based learning. Tensor network is chosen since it is the best platform to introduce thermal fluctuation. Different from traditional network structure which directly takes the summation of individual losses as its loss function, FTTN regards it as thermal average loss computed from the entanglement with the environment. The temperature-like parameter can be automatically optimized, which gives each database an individual temperature. FTTN obtains improvement in both test accuracy and convergence speed in several datasets. The non-zero temperature automatically separates similar features, avoiding the wrong classification in previous architecture. The thermal fluctuation may give a better improvement in other frameworks, and we may also implement the temperature of database to improve the training effect.

Related papers

• Optimizing Temperature for Language Models with Multi-Sample Inference [47.14991144052361]
  This paper addresses the challenge of automatically identifying the (near)-optimal temperature for different large language models. We provide a comprehensive analysis of temperature's role in performance optimization, considering variations in model architectures, datasets, task types, model sizes, and predictive accuracy. We propose a novel entropy-based metric for automated temperature optimization, which consistently outperforms fixed-temperature baselines.
  arXiv  Detail & Related papers  (2025-02-07T19:35:25Z)
• Thermal Vision: Pioneering Non-Invasive Temperature Tracking in Congested Spaces [0.0]
  Existing research on non-invasive temperature estimation using thermal cameras has predominantly focused on sparse settings. Our study proposes a non-invasive temperature estimation system that combines a thermal camera with an edge device. Our proposed face detection model achieves an impressive mAP score of over 84 in both in-dataset and cross-dataset evaluations.
  arXiv  Detail & Related papers  (2024-12-01T15:52:51Z)
• Adaptive Decoding via Latent Preference Optimization [55.70602730588745]
  We introduce Adaptive Decoding, a layer added to the model to select the sampling temperature dynamically at inference time. Our method outperforms all fixed decoding temperatures across a range of tasks that require different temperatures.
  arXiv  Detail & Related papers  (2024-11-14T18:31:39Z)
• A finite element-based physics-informed operator learning framework for spatiotemporal partial differential equations on arbitrary domains [33.7054351451505]
  We propose a novel finite element-based physics operator learning framework that allows for predicting dynamics governed by partial differential equations (PDEs) The proposed operator learning framework takes a temperature field at the current time step as input and predicts a temperature field at the next time step. Networks successfully predict the temperature evolution over time for any initial temperature field at high accuracy compared to the FEM solution.
  arXiv  Detail & Related papers  (2024-05-21T02:41:40Z)
• Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training [58.20089993899729]
  This paper proposes TempBalance, a straightforward yet effective layerwise learning rate method. We show that TempBalance significantly outperforms ordinary SGD and carefully-tuned spectral norm regularization. We also show that TempBalance outperforms a number of state-of-the-art metrics and schedulers.
  arXiv  Detail & Related papers  (2023-12-01T05:38:17Z)
• Small Temperature is All You Need for Differentiable Architecture Search [8.93957397187611]
  Differentiable architecture search (DARTS) yields highly efficient gradient-based neural architecture search (NAS) We propose to close the gap between the relaxed supernet in training and the pruned finalnet in evaluation through utilizing small temperature.
  arXiv  Detail & Related papers  (2023-06-12T04:01:57Z)
• Not All Semantics are Created Equal: Contrastive Self-supervised Learning with Automatic Temperature Individualization [51.41175648612714]
  We propose a new robust contrastive loss inspired by distributionally robust optimization (DRO) We show that our algorithm automatically learns a suitable $tau$ for each sample. Our method outperforms prior strong baselines on unimodal and bimodal datasets.
  arXiv  Detail & Related papers  (2023-05-19T19:25:56Z)
• Fine-tune your Classifier: Finding Correlations With Temperature [2.071516130824992]
  We analyze the impact of temperature on classification tasks by describing a dataset as a set of statistics computed on representations. We study the correlation between these extracted statistics and the observed optimal temperatures.
  arXiv  Detail & Related papers  (2022-10-18T09:48:46Z)
• Energy-Efficient Model Compression and Splitting for Collaborative Inference Over Time-Varying Channels [52.60092598312894]
  We propose a technique to reduce the total energy bill at the edge device by utilizing model compression and time-varying model split between the edge and remote nodes. Our proposed solution results in minimal energy consumption and $CO$ emission compared to the considered baselines.
  arXiv  Detail & Related papers  (2021-06-02T07:36:27Z)
• Critical Phenomena in Complex Networks: from Scale-free to Random Networks [77.34726150561087]
  We study critical phenomena in a class of configuration network models with hidden variables controlling links between pairs of nodes. We find analytical expressions for the average node degree, the expected number of edges, and the Landau and Helmholtz free energies, as a function of the temperature and number of nodes.
  arXiv  Detail & Related papers  (2020-08-05T18:57:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.