Information flow in multilayer perceptrons: an in-depth analysis

• URL: http://arxiv.org/abs/2510.13846v1
• Date: Sat, 11 Oct 2025 19:38:06 GMT
• Title: Information flow in multilayer perceptrons: an in-depth analysis
• Abstract summary: Analysing how information flows along the layers of a multilayer perceptron is a topic of paramount importance in the field of artificial neural networks.<n>The concept of information matrix is devised and then used as formal framework for understanding the aetiology of optimisation strategies.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Analysing how information flows along the layers of a multilayer perceptron is a topic of paramount importance in the field of artificial neural networks. After framing the problem from the point of view of information theory, in this position article a specific investigation is conducted on the way information is processed, with particular reference to the requirements imposed by supervised learning. To this end, the concept of information matrix is devised and then used as formal framework for understanding the aetiology of optimisation strategies and for studying the information flow. The underlying research for this article has also produced several key outcomes: i) the definition of a parametric optimisation strategy, ii) the finding that the optimisation strategy proposed in the information bottleneck framework shares strong similarities with the one derived from the information matrix, and iii) the insight that a multilayer perceptron serves as a kind of "adaptor", meant to process the input according to the given objective.

Related papers

• VISTA: Visualization of Token Attribution via Efficient Analysis [0.3227406536443669]
  We introduce a model-agnostic token importance visualization technique to better understand how generative AI systems perceive and prioritize information from input text.<n>Our method leverages perturbation-based strategies combined with a three-matrix analytical framework to generate relevance maps.<n>We derive a composite importance score that provides a nuanced and mathematically grounded measure of token significance.
  arXiv  Detail & Related papers  (2026-04-02T16:09:13Z)
• How do Large Language Models Understand Relevance? A Mechanistic Interpretability Perspective [64.00022624183781]
  Large language models (LLMs) can assess relevance and support information retrieval (IR) tasks.<n>We investigate how different LLM modules contribute to relevance judgment through the lens of mechanistic interpretability.
  arXiv  Detail & Related papers  (2025-04-10T16:14:55Z)
• Enhancing Neural Network Interpretability Through Conductance-Based Information Plane Analysis [0.0]
  The Information Plane is a conceptual framework used to analyze the flow of information in neural networks. This paper introduces a new approach that uses layer conductance, a measure of sensitivity to input features, to enhance the Information Plane analysis.
  arXiv  Detail & Related papers  (2024-08-26T23:10:42Z)
• Coding for Intelligence from the Perspective of Category [66.14012258680992]
  Coding targets compressing and reconstructing data, and intelligence. Recent trends demonstrate the potential homogeneity of these two fields. We propose a novel problem of Coding for Intelligence from the category theory view.
  arXiv  Detail & Related papers  (2024-07-01T07:05:44Z)
• Deciphering 'What' and 'Where' Visual Pathways from Spectral Clustering of Layer-Distributed Neural Representations [15.59251297818324]
  We present an approach for analyzing grouping information contained within a neural network's activations. We exploit features from all layers and obviating the need to guess which part of the model contains relevant information.
  arXiv  Detail & Related papers  (2023-12-11T01:20:34Z)
• Embedding in Recommender Systems: A Survey [54.55152033023537]
  This survey presents a comprehensive analysis of advances in recommender system embedding techniques.<n>In matrix-based scenarios, collaborative filtering generates embeddings that effectively model user-item preferences.<n>We introduce emerging approaches, including AutoML, hashing techniques, and quantization methods, to enhance performance.
  arXiv  Detail & Related papers  (2023-10-28T06:31:06Z)
• To Compress or Not to Compress- Self-Supervised Learning and Information Theory: A Review [30.87092042943743]
  Deep neural networks excel in supervised learning tasks but are constrained by the need for extensive labeled data. Self-supervised learning emerges as a promising alternative, allowing models to learn without explicit labels. Information theory, and notably the information bottleneck principle, has been pivotal in shaping deep neural networks.
  arXiv  Detail & Related papers  (2023-04-19T00:33:59Z)
• Deep networks for system identification: a Survey [56.34005280792013]
  System identification learns mathematical descriptions of dynamic systems from input-output data. Main aim of the identified model is to predict new data from previous observations. We discuss architectures commonly adopted in the literature, like feedforward, convolutional, and recurrent networks.
  arXiv  Detail & Related papers  (2023-01-30T12:38:31Z)
• Variational Distillation for Multi-View Learning [104.17551354374821]
  We design several variational information bottlenecks to exploit two key characteristics for multi-view representation learning. Under rigorously theoretical guarantee, our approach enables IB to grasp the intrinsic correlation between observations and semantic labels.
  arXiv  Detail & Related papers  (2022-06-20T03:09:46Z)
• Information-Theoretic Odometry Learning [83.36195426897768]
  We propose a unified information theoretic framework for learning-motivated methods aimed at odometry estimation. The proposed framework provides an elegant tool for performance evaluation and understanding in information-theoretic language.
  arXiv  Detail & Related papers  (2022-03-11T02:37:35Z)
• Inter-layer Information Similarity Assessment of Deep Neural Networks Via Topological Similarity and Persistence Analysis of Data Neighbour Dynamics [93.4221402881609]
  The quantitative analysis of information structure through a deep neural network (DNN) can unveil new insights into the theoretical performance of DNN architectures. Inspired by both LS and ID strategies for quantitative information structure analysis, we introduce two novel complimentary methods for inter-layer information similarity assessment. We demonstrate their efficacy in this study by performing analysis on a deep convolutional neural network architecture on image data.
  arXiv  Detail & Related papers  (2020-12-07T15:34:58Z)
• Unsupervised Domain Adaptation in Semantic Segmentation: a Review [22.366638308792734]
  The aim of this paper is to give an overview of the recent advancements in the Unsupervised Domain Adaptation (UDA) of deep networks for semantic segmentation.
  arXiv  Detail & Related papers  (2020-05-21T20:10:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.