The Information-Theoretic Imperative: Compression and the Epistemic Foundations of Intelligence

• URL: http://arxiv.org/abs/2510.25883v1
• Date: Wed, 29 Oct 2025 18:28:06 GMT
• Title: The Information-Theoretic Imperative: Compression and the Epistemic Foundations of Intelligence
• Authors: Christian Dittrich, Jennifer Flygare Kinne,
• Abstract summary: Existing frameworks converge on the centrality of compression to intelligence but leave underspecified why this process enforces the discovery of causal structure.<n>We introduce a two-level framework to address this gap.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing frameworks converge on the centrality of compression to intelligence but leave underspecified why this process enforces the discovery of causal structure rather than superficial statistical patterns. We introduce a two-level framework to address this gap. The Information-Theoretic Imperative (ITI) establishes that any system persisting in uncertain environments must minimize epistemic entropy through predictive compression: this is the evolutionary "why" linking survival pressure to information-processing demands. The Compression Efficiency Principle (CEP) specifies how efficient compression mechanically selects for generative, causal models through exception-accumulation dynamics, making reality alignment a consequence rather than a contingent achievement. Together, ITI and CEP define a causal chain: from survival pressure to prediction necessity, compression requirement, efficiency optimization, generative structure discovery, and ultimately reality alignment. Each link follows from physical, information-theoretic, or evolutionary constraints, implying that intelligence is the mechanically necessary outcome of persistence in structured environments. This framework yields empirically testable predictions: compression efficiency, measured as approach to the rate-distortion frontier, correlates with out-of-distribution generalization; exception-accumulation rates differentiate causal from correlational models; hierarchical systems exhibit increasing efficiency across abstraction layers; and biological systems demonstrate metabolic costs that track representational complexity. ITI and CEP thereby provide a unified account of convergence across biological, artificial, and multi-scale systems, addressing the epistemic and functional dimensions of intelligence without invoking assumptions about consciousness or subjective experience.

Related papers

• The ASIR Courage Model: A Phase-Dynamic Framework for Truth Transitions in Human and AI Systems [1.4524117432184773]
  ASIR Courage Model formalizes truth-disclosure as a state transition rather than a personality trait.<n>Phase-dynamic framework initially formulated for human truth-telling under asymmetric stakes.<n>Same architecture extends to AI systems operating under policy constraints and alignment filters.
  arXiv  Detail & Related papers  (2026-02-25T09:56:26Z)
• Structured Hybrid Mechanistic Models for Robust Estimation of Time-Dependent Intervention Outcomes [9.820469663506882]
  Estimating intervention effects in dynamical systems is crucial for outcome optimization.<n>Mechanistic models are typically robust, but might be oversimplified.<n>We propose a hybrid mechanistic-data-driven approach to estimate time-dependent intervention outcomes.
  arXiv  Detail & Related papers  (2026-02-11T20:39:41Z)
• Toward a Physical Theory of Intelligence [0.016144088896423884]
  We present a theory of intelligence grounded in irreversible information processing in systems constrained by conservation laws.<n>An intelligent system is modelled as a coupled agent-environment process whose evolution transforms information into goal-directed work.
  arXiv  Detail & Related papers  (2025-12-22T20:40:27Z)
• Knowledge-Informed Neural Network for Complex-Valued SAR Image Recognition [51.03674130115878]
  We introduce the Knowledge-Informed Neural Network (KINN), a lightweight framework built upon a novel "compression-aggregation-compression" architecture.<n>KINN establishes a state-of-the-art in parameter-efficient recognition, offering exceptional generalization in data-scarce and out-of-distribution scenarios.
  arXiv  Detail & Related papers  (2025-10-23T07:12:26Z)
• On the Interaction of Compressibility and Adversarial Robustness [25.58735050707295]
  We analyze how different forms of compressibility - such as neuron-level sparsity and spectral compressibility - affect adversarial robustness.<n>We show that these forms of compression can induce a small number of highly sensitive directions in the representation space, which adversaries can exploit.<n>Our findings show a fundamental tension between structured compressibility and robustness, and suggest new pathways for designing models that are both efficient and secure.
  arXiv  Detail & Related papers  (2025-07-23T17:35:48Z)
• Algorithmic causal structure emerging through compression [53.52699766206808]
  We explore the relationship between causality, symmetry, and compression.<n>We build on and generalize the known connection between learning and compression to a setting where causal models are not identifiable.<n>We define algorithmic causality as an alternative definition of causality when traditional assumptions for causal identifiability do not hold.
  arXiv  Detail & Related papers  (2025-02-06T16:50:57Z)
• An Information-Theoretic Regularizer for Lossy Neural Image Compression [20.939331919455935]
  Lossy image compression networks aim to minimize the latent entropy of images while adhering to specific distortion constraints.<n>We propose a novel structural regularization method for the neural image compression task by incorporating the negative conditional source entropy into the training objective.
  arXiv  Detail & Related papers  (2024-11-23T05:19:27Z)
• Causal Representation Learning from Multimodal Biomedical Observations [57.00712157758845]
  We develop flexible identification conditions for multimodal data and principled methods to facilitate the understanding of biomedical datasets.<n>Key theoretical contribution is the structural sparsity of causal connections between modalities.<n>Results on a real-world human phenotype dataset are consistent with established biomedical research.
  arXiv  Detail & Related papers  (2024-11-10T16:40:27Z)
• Disentangling the Causes of Plasticity Loss in Neural Networks [55.23250269007988]
  We show that loss of plasticity can be decomposed into multiple independent mechanisms. We show that a combination of layer normalization and weight decay is highly effective at maintaining plasticity in a variety of synthetic nonstationary learning tasks.
  arXiv  Detail & Related papers  (2024-02-29T00:02:33Z)
• Targeted Reduction of Causal Models [55.11778726095353]
  Causal Representation Learning offers a promising avenue to uncover interpretable causal patterns in simulations. We introduce Targeted Causal Reduction (TCR), a method for condensing complex intervenable models into a concise set of causal factors. Its ability to generate interpretable high-level explanations from complex models is demonstrated on toy and mechanical systems.
  arXiv  Detail & Related papers  (2023-11-30T15:46:22Z)
• Spectral chaos bounds from scaling theory of maximally efficient quantum-dynamical scrambling [44.99833362998488]
  A key conjecture about the evolution of complex quantum systems towards an ergodic steady state, known as scrambling, is that this process acquires universal features when it is most efficient.<n>We develop a single- parameter scaling theory for the spectral statistics in this scenario, which embodies exact self-similarity of the spectral correlations along the complete scrambling dynamics.<n>We establish that scaling predictions are matched by a privileged process and serve as bounds for other dynamical scrambling scenarios, allowing one to quantify inefficient or incomplete scrambling on all time scales.
  arXiv  Detail & Related papers  (2023-10-17T15:41:50Z)
• Discovering Latent Causal Variables via Mechanism Sparsity: A New Principle for Nonlinear ICA [81.4991350761909]
  Independent component analysis (ICA) refers to an ensemble of methods which formalize this goal and provide estimation procedure for practical application. We show that the latent variables can be recovered up to a permutation if one regularizes the latent mechanisms to be sparse.
  arXiv  Detail & Related papers  (2021-07-21T14:22:14Z)
• The Causal Neural Connection: Expressiveness, Learnability, and Inference [125.57815987218756]
  An object called structural causal model (SCM) represents a collection of mechanisms and sources of random variation of the system under investigation. In this paper, we show that the causal hierarchy theorem (Thm. 1, Bareinboim et al., 2020) still holds for neural models. We introduce a special type of SCM called a neural causal model (NCM), and formalize a new type of inductive bias to encode structural constraints necessary for performing causal inferences.
  arXiv  Detail & Related papers  (2021-07-02T01:55:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.