Grokking as an entanglement transition in tensor network machine learning

• URL: http://arxiv.org/abs/2503.10483v1
• Date: Thu, 13 Mar 2025 15:51:23 GMT
• Title: Grokking as an entanglement transition in tensor network machine learning
• Authors: Domenico Pomarico, Alfonso Monaco, Giuseppe Magnifico, Antonio Lacalamita, Ester Pantaleo, Loredana Bellantuono, Sabina Tangaro, Tommaso Maggipinto, Marianna La Rocca, Ernesto Picardi, Nicola Amoroso, Graziano Pesole, Sebastiano Stramaglia, Roberto Bellotti,
• Abstract summary: We numerically prove that grokking phenomenon can be related to an entanglement dynamical transition in the underlying quantum many-body systems.<n>We exploit measurement of qubits magnetization and correlation functions in the Matrix Product State network as a tool to identify meaningful and relevant gene subcommunities.
• Score: 0.608657548424657
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Grokking is a intriguing phenomenon in machine learning where a neural network, after many training iterations with negligible improvement in generalization, suddenly achieves high accuracy on unseen data. By working in the quantum-inspired machine learning framework based on tensor networks, we numerically prove that grokking phenomenon can be related to an entanglement dynamical transition in the underlying quantum many-body systems, consisting in a one-dimensional lattice with each site hosting a qubit. Two datasets are considered as use case scenarios, namely fashion MNIST and gene expression communities of hepatocellular carcinoma. In both cases, we train Matrix Product State (MPS) to perform binary classification tasks, and we analyse the learning dynamics. We exploit measurement of qubits magnetization and correlation functions in the MPS network as a tool to identify meaningful and relevant gene subcommunities, verified by means of enrichment procedures.

Related papers

• Transfer entropy and O-information to detect grokking in tensor network multi-class classification problems [0.6222849132943892]
  We study the training dynamics of Matrix Product State (MPS) classifiers applied to three-class problems.<n>We investigate the phenomenon of grokking, where generalization emerges suddenly after memorization.<n>Our results show that grokking in the fashion MNIST task coincides with a sharp entanglement transition and a peak in redundant information, whereas the overfitted hyper-spectral model retains synergistic, disordered behavior.
  arXiv  Detail & Related papers  (2025-07-31T08:53:04Z)
• Dissipation-driven quantum generative adversarial networks [11.833077116494929]
  We introduce a novel dissipation-driven quantum generative adversarial network (DQGAN) architecture specifically tailored for generating classical data. The classical data is encoded into the input qubits of the input layer via strong tailored dissipation processes. We extract both the generated data and the classification results by measuring the observables of the steady state of the output qubits.
  arXiv  Detail & Related papers  (2024-08-28T07:41:58Z)
• A Matrix Product State Model for Simultaneous Classification and Generation [0.8192907805418583]
  Quantum machine learning (QML) is a rapidly expanding field that merges the principles of quantum computing with the techniques of machine learning. Here, we present a novel matrix product state (MPS) model, where the MPS functions as both a classifier and a generator. Our contributions offer insights into the mechanics of tensor network methods for generation tasks.
  arXiv  Detail & Related papers  (2024-06-25T10:23:36Z)
• Assessing Neural Network Representations During Training Using Noise-Resilient Diffusion Spectral Entropy [55.014926694758195]
  Entropy and mutual information in neural networks provide rich information on the learning process. We leverage data geometry to access the underlying manifold and reliably compute these information-theoretic measures. We show that they form noise-resistant measures of intrinsic dimensionality and relationship strength in high-dimensional simulated data.
  arXiv  Detail & Related papers  (2023-12-04T01:32:42Z)
• Heterogenous Memory Augmented Neural Networks [84.29338268789684]
  We introduce a novel heterogeneous memory augmentation approach for neural networks. By introducing learnable memory tokens with attention mechanism, we can effectively boost performance without huge computational overhead. We show our approach on various image and graph-based tasks under both in-distribution (ID) and out-of-distribution (OOD) conditions.
  arXiv  Detail & Related papers  (2023-10-17T01:05:28Z)
• Learning Multiscale Consistency for Self-supervised Electron Microscopy Instance Segmentation [48.267001230607306]
  We propose a pretraining framework that enhances multiscale consistency in EM volumes. Our approach leverages a Siamese network architecture, integrating strong and weak data augmentations. It effectively captures voxel and feature consistency, showing promise for learning transferable representations for EM analysis.
  arXiv  Detail & Related papers  (2023-08-19T05:49:13Z)
• Unsupervised Learning of Invariance Transformations [105.54048699217668]
  We develop an algorithmic framework for finding approximate graph automorphisms. We discuss how this framework can be used to find approximate automorphisms in weighted graphs in general.
  arXiv  Detail & Related papers  (2023-07-24T17:03:28Z)
• Fluctuation based interpretable analysis scheme for quantum many-body snapshots [0.0]
  Microscopically understanding and classifying phases of matter is at the heart of strongly-correlated quantum physics. Here, we combine confusion learning with correlation convolutional neural networks, which yields fully interpretable phase detection. Our work opens new directions in interpretable quantum image processing being sensible to long-range order.
  arXiv  Detail & Related papers  (2023-04-12T17:59:59Z)
• A didactic approach to quantum machine learning with a single qubit [68.8204255655161]
  We focus on the case of learning with a single qubit, using data re-uploading techniques. We implement the different proposed formulations in toy and real-world datasets using the qiskit quantum computing SDK.
  arXiv  Detail & Related papers  (2022-11-23T18:25:32Z)
• Deep learning of spatial densities in inhomogeneous correlated quantum systems [0.0]
  We show that we can learn to predict densities using convolutional neural networks trained on random potentials. We show that our approach can handle well the interplay of interference and interactions and the behaviour of models with phase transitions in inhomogeneous situations.
  arXiv  Detail & Related papers  (2022-11-16T17:10:07Z)
• Identification of quantum entanglement with Siamese convolutional neural networks and semi-supervised learning [0.0]
  Quantum entanglement is a fundamental property commonly used in various quantum information protocols and algorithms. In this study, we use deep convolutional NNs, a type of supervised machine learning, to identify quantum entanglement for any bi Partition in a 3-qubit system.
  arXiv  Detail & Related papers  (2022-10-13T23:17:55Z)
• Decomposing neural networks as mappings of correlation functions [57.52754806616669]
  We study the mapping between probability distributions implemented by a deep feed-forward network. We identify essential statistics in the data, as well as different information representations that can be used by neural networks.
  arXiv  Detail & Related papers  (2022-02-10T09:30:31Z)
• Data-driven emergence of convolutional structure in neural networks [83.4920717252233]
  We show how fully-connected neural networks solving a discrimination task can learn a convolutional structure directly from their inputs. By carefully designing data models, we show that the emergence of this pattern is triggered by the non-Gaussian, higher-order local structure of the inputs.
  arXiv  Detail & Related papers  (2022-02-01T17:11:13Z)
• Quantum-inspired event reconstruction with Tensor Networks: Matrix Product States [0.0]
  We show that Networks are ideal vehicles to connect quantum mechanical concepts to machine learning techniques. We show that entanglement entropy can be used to interpret what a network learns.
  arXiv  Detail & Related papers  (2021-06-15T18:00:02Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.