The Information-Theoretic Imperative: Compression and the Epistemic Foundations of Intelligence

• URL: http://arxiv.org/abs/2510.25883v1
• Date: Wed, 29 Oct 2025 18:28:06 GMT
• Title: The Information-Theoretic Imperative: Compression and the Epistemic Foundations of Intelligence
• Abstract summary: Existing frameworks converge on the centrality of compression to intelligence but leave underspecified why this process enforces the discovery of causal structure.<n>We introduce a two-level framework to address this gap.
• Score: 0.0
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Existing frameworks converge on the centrality of compression to intelligence but leave underspecified why this process enforces the discovery of causal structure rather than superficial statistical patterns. We introduce a two-level framework to address this gap. The Information-Theoretic Imperative (ITI) establishes that any system persisting in uncertain environments must minimize epistemic entropy through predictive compression: this is the evolutionary "why" linking survival pressure to information-processing demands. The Compression Efficiency Principle (CEP) specifies how efficient compression mechanically selects for generative, causal models through exception-accumulation dynamics, making reality alignment a consequence rather than a contingent achievement. Together, ITI and CEP define a causal chain: from survival pressure to prediction necessity, compression requirement, efficiency optimization, generative structure discovery, and ultimately reality alignment. Each link follows from physical, information-theoretic, or evolutionary constraints, implying that intelligence is the mechanically necessary outcome of persistence in structured environments. This framework yields empirically testable predictions: compression efficiency, measured as approach to the rate-distortion frontier, correlates with out-of-distribution generalization; exception-accumulation rates differentiate causal from correlational models; hierarchical systems exhibit increasing efficiency across abstraction layers; and biological systems demonstrate metabolic costs that track representational complexity. ITI and CEP thereby provide a unified account of convergence across biological, artificial, and multi-scale systems, addressing the epistemic and functional dimensions of intelligence without invoking assumptions about consciousness or subjective experience.

Related papers

• Normative Networks for Source Separation via Local Plasticity and Dendritic Computation [8.607903455924562]
  Blind source separation (BSS) is a framework for studying how latent causes may be recovered from sensory mixtures.<n>We propose Predictive Entropy Maximization, which achieves competitive performance in BSS, using only local weight updates.
  arXiv  Detail & Related papers  (2026-05-19T15:15:26Z)
• Partial Effective Information Decomposition for Synergistic Causality [7.157956561479179]
  This paper proposes a framework that decomposes the influence of multiple source variables on a target variable under maximum-entropy interventions.<n>Based on this framework, it is possible to define causal graphs containing hyperedges as well as downward causation.<n>Applying the framework to a machine-learning-based air quality forecasting task on KnowAir-V2, we demonstrate that PEID can extract interpretable inter-station causal structures.
  arXiv  Detail & Related papers  (2026-05-05T01:39:17Z)
• Information as Maximum-Caliber Deviation: A bridge between Integrated Information Theory and the Free Energy Principle [51.56484100374058]
  This work proposes that information can be defined as the deviation $$ of realized dynamics from a constrained maximum-caliber (MaxCal) path ensemble over a finite time horizon.<n>Under this definition, each of the cause/effect repertoires central to IIT 3.0 emerge directly from MaxCal variational principles.<n>Information $$ is shown to be equivalent to prediction error under accompanying predictive coding models.
  arXiv  Detail & Related papers  (2026-05-03T07:22:55Z)
• PACE: Post-Causal Entropy Modeling for Learned LiDAR Point Cloud Compression [43.524231862354405]
  LiDAR point cloud compression is vital for autonomous systems to handle massive data from high-resolution sensors.<n>We propose PACE, a new framework that reformulates ancestral context aggregation as a non-causal backbone and causality to a lightweight, stage-scalable predictor.<n>Experiments demonstrate that PACE sets a new state-of-the-art in compression efficiency, achieving notable BD-BR savings and reducing decoding latency by over 90% in autoregressive mode.
  arXiv  Detail & Related papers  (2026-05-02T08:36:11Z)
• Entropic Claim Resolution: Uncertainty-Driven Evidence Selection for RAG [0.0]
  We introduce Entropic Claim Resolution (ECR), a novel inference-time algorithm that reframes RAG reasoning as entropy minimization.<n>ECR selects atomic evidence claims by maximizing Expected Entropy Reduction (EER), a decision-theoretic criterion for the value of information.<n>Our framework provides a rigorous foundation for uncertainty-aware evidence selection, shifting the paradigm from retrieving what is most relevant to retrieving what is most discriminative.
  arXiv  Detail & Related papers  (2026-03-30T13:49:03Z)
• The ASIR Courage Model: A Phase-Dynamic Framework for Truth Transitions in Human and AI Systems [1.4524117432184773]
  ASIR Courage Model formalizes truth-disclosure as a state transition rather than a personality trait.<n>Phase-dynamic framework initially formulated for human truth-telling under asymmetric stakes.<n>Same architecture extends to AI systems operating under policy constraints and alignment filters.
  arXiv  Detail & Related papers  (2026-02-25T09:56:26Z)
• Structured Hybrid Mechanistic Models for Robust Estimation of Time-Dependent Intervention Outcomes [9.820469663506882]
  Estimating intervention effects in dynamical systems is crucial for outcome optimization.<n>Mechanistic models are typically robust, but might be oversimplified.<n>We propose a hybrid mechanistic-data-driven approach to estimate time-dependent intervention outcomes.
  arXiv  Detail & Related papers  (2026-02-11T20:39:41Z)
• Toward a Physical Theory of Intelligence [0.016144088896423884]
  We present a theory of intelligence grounded in irreversible information processing in systems constrained by conservation laws.<n>An intelligent system is modelled as a coupled agent-environment process whose evolution transforms information into goal-directed work.
  arXiv  Detail & Related papers  (2025-12-22T20:40:27Z)
• BITS for GAPS: Bayesian Information-Theoretic Sampling for hierarchical GAussian Process Surrogates [45.88028371034407]
  We introduce the Bayesian Information-Theoretic Sampling for hierarchical GAussian Process Surrogates (BITS for GAPS) framework.<n>BITS for GAPS supports serial hybrid modeling, where known physics governs part of the system.<n>We derive entropy-based acquisition functions that quantify expected information gain from candidate input locations.
  arXiv  Detail & Related papers  (2025-11-20T21:36:21Z)
• Knowledge-Informed Neural Network for Complex-Valued SAR Image Recognition [51.03674130115878]
  We introduce the Knowledge-Informed Neural Network (KINN), a lightweight framework built upon a novel "compression-aggregation-compression" architecture.<n>KINN establishes a state-of-the-art in parameter-efficient recognition, offering exceptional generalization in data-scarce and out-of-distribution scenarios.
  arXiv  Detail & Related papers  (2025-10-23T07:12:26Z)
• On the Interaction of Compressibility and Adversarial Robustness [25.58735050707295]
  We analyze how different forms of compressibility - such as neuron-level sparsity and spectral compressibility - affect adversarial robustness.<n>We show that these forms of compression can induce a small number of highly sensitive directions in the representation space, which adversaries can exploit.<n>Our findings show a fundamental tension between structured compressibility and robustness, and suggest new pathways for designing models that are both efficient and secure.
  arXiv  Detail & Related papers  (2025-07-23T17:35:48Z)
• Algorithmic causal structure emerging through compression [53.52699766206808]
  We explore the relationship between causality, symmetry, and compression.<n>We build on and generalize the known connection between learning and compression to a setting where causal models are not identifiable.<n>We define algorithmic causality as an alternative definition of causality when traditional assumptions for causal identifiability do not hold.
  arXiv  Detail & Related papers  (2025-02-06T16:50:57Z)
• An Information-Theoretic Regularizer for Lossy Neural Image Compression [20.939331919455935]
  Lossy image compression networks aim to minimize the latent entropy of images while adhering to specific distortion constraints.<n>We propose a novel structural regularization method for the neural image compression task by incorporating the negative conditional source entropy into the training objective.
  arXiv  Detail & Related papers  (2024-11-23T05:19:27Z)
• Causal Representation Learning from Multimodal Biomedical Observations [57.00712157758845]
  We develop flexible identification conditions for multimodal data and principled methods to facilitate the understanding of biomedical datasets.<n>Key theoretical contribution is the structural sparsity of causal connections between modalities.<n>Results on a real-world human phenotype dataset are consistent with established biomedical research.
  arXiv  Detail & Related papers  (2024-11-10T16:40:27Z)
• Disentangling the Causes of Plasticity Loss in Neural Networks [55.23250269007988]
  We show that loss of plasticity can be decomposed into multiple independent mechanisms. We show that a combination of layer normalization and weight decay is highly effective at maintaining plasticity in a variety of synthetic nonstationary learning tasks.
  arXiv  Detail & Related papers  (2024-02-29T00:02:33Z)
• Targeted Reduction of Causal Models [55.11778726095353]
  Causal Representation Learning offers a promising avenue to uncover interpretable causal patterns in simulations. We introduce Targeted Causal Reduction (TCR), a method for condensing complex intervenable models into a concise set of causal factors. Its ability to generate interpretable high-level explanations from complex models is demonstrated on toy and mechanical systems.
  arXiv  Detail & Related papers  (2023-11-30T15:46:22Z)
• Spectral chaos bounds from scaling theory of maximally efficient quantum-dynamical scrambling [44.99833362998488]
  A key conjecture about the evolution of complex quantum systems towards an ergodic steady state, known as scrambling, is that this process acquires universal features when it is most efficient.<n>We develop a single- parameter scaling theory for the spectral statistics in this scenario, which embodies exact self-similarity of the spectral correlations along the complete scrambling dynamics.<n>We establish that scaling predictions are matched by a privileged process and serve as bounds for other dynamical scrambling scenarios, allowing one to quantify inefficient or incomplete scrambling on all time scales.
  arXiv  Detail & Related papers  (2023-10-17T15:41:50Z)
• Discovering Latent Causal Variables via Mechanism Sparsity: A New Principle for Nonlinear ICA [81.4991350761909]
  Independent component analysis (ICA) refers to an ensemble of methods which formalize this goal and provide estimation procedure for practical application. We show that the latent variables can be recovered up to a permutation if one regularizes the latent mechanisms to be sparse.
  arXiv  Detail & Related papers  (2021-07-21T14:22:14Z)
• The Causal Neural Connection: Expressiveness, Learnability, and Inference [125.57815987218756]
  An object called structural causal model (SCM) represents a collection of mechanisms and sources of random variation of the system under investigation. In this paper, we show that the causal hierarchy theorem (Thm. 1, Bareinboim et al., 2020) still holds for neural models. We introduce a special type of SCM called a neural causal model (NCM), and formalize a new type of inductive bias to encode structural constraints necessary for performing causal inferences.
  arXiv  Detail & Related papers  (2021-07-02T01:55:18Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.