A New Information Theory of Certainty for Machine Learning

• URL: http://arxiv.org/abs/2304.12833v1
• Date: Tue, 25 Apr 2023 14:03:57 GMT
• Title: A New Information Theory of Certainty for Machine Learning
• Authors: Arthur Jun Zhang
• Abstract summary: Claude Shannon coined entropy to quantify the uncertainty of a random distribution for communication coding theory. We propose a new concept troenpy,as the canonical dual of entropy, to quantify the certainty of the underlying distribution.
• Score: 0.0
• License: http://creativecommons.org/licenses/by-nc-nd/4.0/
• Abstract: Claude Shannon coined entropy to quantify the uncertainty of a random distribution for communication coding theory. We observe that the uncertainty nature of entropy also limits its direct usage in mathematical modeling. Therefore we propose a new concept troenpy,as the canonical dual of entropy, to quantify the certainty of the underlying distribution. We demonstrate two applications in machine learning. The first is for the classical document classification, we develop a troenpy based weighting scheme to leverage the document class label. The second is a self-troenpy weighting scheme for sequential data and show that it can be easily included in neural network based language models and achieve dramatic perplexity reduction. We also define quantum troenpy as the dual of the Von Neumann entropy to quantify the certainty of quantum systems.

Related papers

• ShadowNet for Data-Centric Quantum System Learning [188.683909185536]
  We propose a data-centric learning paradigm combining the strength of neural-network protocols and classical shadows. Capitalizing on the generalization power of neural networks, this paradigm can be trained offline and excel at predicting previously unseen systems. We present the instantiation of our paradigm in quantum state tomography and direct fidelity estimation tasks and conduct numerical analysis up to 60 qubits.
  arXiv  Detail & Related papers  (2023-08-22T09:11:53Z)
• The Expressivity of Classical and Quantum Neural Networks on Entanglement Entropy [0.3299672391663526]
  Von Neumann entropy from R'enyi entropies is a challenging problem in quantum field theory. We propose a general framework to tackle this problem using classical and quantum neural networks with supervised learning. Our proposed methods can accurately predict the von Neumann and R'enyi entropies numerically.
  arXiv  Detail & Related papers  (2023-05-01T18:00:01Z)
• Quantum Conformal Prediction for Reliable Uncertainty Quantification in Quantum Machine Learning [47.991114317813555]
  Quantum models implement implicit probabilistic predictors that produce multiple random decisions for each input through measurement shots. This paper proposes to leverage such randomness to define prediction sets for both classification and regression that provably capture the uncertainty of the model.
  arXiv  Detail & Related papers  (2023-04-06T22:05:21Z)
• A new entanglement measure based dual entropy [7.95085289592294]
  We define $St$-entropy entanglement based on von Neumann entropy and its complementary dual. We prove a new type of entanglement inequality in terms of $St$-entropy entanglement for quantum entangled networks.
  arXiv  Detail & Related papers  (2022-04-15T10:08:12Z)
• Diagnosis of information scrambling from Hamiltonian evolution under decoherence [1.5791732557395552]
  We apply a quantum teleportation protocol based on the Hayden-Preskill thought experiment to quantify how scrambling a given quantum evolution is. We numerically simulate the time evolution of the two theories in the Hamiltonian formalism. We find that the Yang-Mills-Ising model shows the signal of information scrambling at late times.
  arXiv  Detail & Related papers  (2021-03-09T02:14:22Z)
• The Hintons in your Neural Network: a Quantum Field Theory View of Deep Learning [84.33745072274942]
  We show how to represent linear and non-linear layers as unitary quantum gates, and interpret the fundamental excitations of the quantum model as particles. On top of opening a new perspective and techniques for studying neural networks, the quantum formulation is well suited for optical quantum computing.
  arXiv  Detail & Related papers  (2021-03-08T17:24:29Z)
• Single-Atom Verification of the Information-Theoretical Bound of Irreversibility at the Quantum Level [0.11242503819703256]
  In a quantum mechanical fashion, we report the first theoretical prediction and experimental exploration of an information-theoretical bound on the entropy production. Our finding is fundamental to any quantum thermodynamical process and indicates much difference and complexity in quantum thermodynamics with respect to the conventionally classical counterpart.
  arXiv  Detail & Related papers  (2020-07-04T07:20:31Z)
• Parsimonious neural networks learn interpretable physical laws [77.34726150561087]
  We propose parsimonious neural networks (PNNs) that combine neural networks with evolutionary optimization to find models that balance accuracy with parsimony. The power and versatility of the approach is demonstrated by developing models for classical mechanics and to predict the melting temperature of materials from fundamental properties.
  arXiv  Detail & Related papers  (2020-05-08T16:15:47Z)
• Entropy production in the quantum walk [62.997667081978825]
  We focus on the study of the discrete-time quantum walk on the line, from the entropy production perspective. We argue that the evolution of the coin can be modeled as an open two-level system that exchanges energy with the lattice at some effective temperature.
  arXiv  Detail & Related papers  (2020-04-09T23:18:29Z)
• Quantum mechanics is *-algebras and tensor networks [1.479413555822768]
  We provide a systematic approach to quantum mechanics from an information-theoretic perspective. Our formulation needs only a single kind of object, so-called positive *-tensors. We show how various types of models, like real-time evolutions or thermal systems can be translated into *-tensor networks.
  arXiv  Detail & Related papers  (2020-03-17T22:46:00Z)
• Simulation of Thermal Relaxation in Spin Chemistry Systems on a Quantum Computer Using Inherent Qubit Decoherence [53.20999552522241]
  We seek to take advantage of qubit decoherence as a resource in simulating the behavior of real world quantum systems. We present three methods for implementing the thermal relaxation. We find excellent agreement between our results, experimental data, and the theoretical prediction.
  arXiv  Detail & Related papers  (2020-01-03T11:48:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.