Information Science Principles of Machine Learning: A Causal Chain Meta-Framework Based on Formalized Information Mapping

• URL: http://arxiv.org/abs/2505.13182v9
• Date: Fri, 22 Aug 2025 03:53:05 GMT
• Title: Information Science Principles of Machine Learning: A Causal Chain Meta-Framework Based on Formalized Information Mapping
• Authors: Jianfeng Xu,
• Abstract summary: This study addresses key challenges in machine learning, namely the absence of a unified formal theoretical framework and the lack of foundational theories for model interpretability and ethical safety.<n>We first construct a formal information model, explicitly defining the ontological states and carrier mappings of typical machine learning stages.<n>By introducing learnable and processable predicates, as well as learning and processing functions, we analyze the causal chain logic and constraint laws governing machine learning processes.
• Score: 7.299890614172539
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: [Objective] This study addresses key challenges in machine learning, namely the absence of a unified formal theoretical framework and the lack of foundational theories for model interpretability and ethical safety. [Methods] We first construct a formal information model, explicitly defining the ontological states and carrier mappings of typical machine learning stages using sets of well-formed formulas. By introducing learnable and processable predicates, as well as learning and processing functions, we analyze the causal chain logic and constraint laws governing machine learning processes. [Results] We establish the Machine Learning Theory Meta-Framework (MLT-MF), on which we further propose universal definitions for model interpretability and ethical safety. We prove and validate three key theorems: the relationship between model interpretability and information existence, ethical safety assurance, and the upper bound estimation of total variation distance (TVD). [Limitations] The current framework assumes ideal, noise-free information enabling mappings and focuses primarily on model learning and processing logic in static scenarios. It does not yet address information fusion and conflict resolution across ontological spaces in multimodal or multi-agent systems. [Conclusions] This work overcomes the limitations of fragmented research and provides a unified theoretical foundation for systematically addressing critical issues in contemporary machine learning.

Related papers

• The Trinity of Consistency as a Defining Principle for General World Models [106.16462830681452]
  General World Models are capable of learning, simulating, and reasoning about objective physical laws.<n>We propose a principled theoretical framework that defines the essential properties requisite for a General World Model.<n>Our work establishes a principled pathway toward general world models, clarifying both the limitations of current systems and the architectural requirements for future progress.
  arXiv  Detail & Related papers  (2026-02-26T16:15:55Z)
• Towards Worst-Case Guarantees with Scale-Aware Interpretability [58.519943565092724]
  Neural networks organize information according to the hierarchical, multi-scale structure of natural data.<n>We propose a unifying research agenda -- emphscale-aware interpretability -- to develop formal machinery and interpretability tools.
  arXiv  Detail & Related papers  (2026-02-05T01:22:31Z)
• Towards the Formalization of a Trustworthy AI for Mining Interpretable Models explOiting Sophisticated Algorithms [4.587316936127635]
  Interpretable-by-design models are crucial for fostering trust, accountability, and safe adoption of automated decision-making models in real-world applications.<n>We formalize a comprehensive methodology for generating predictive models that balance interpretability with performance.<n>By evaluating ethical measures during model generation, this framework establishes the theoretical foundations for developing AI systems.
  arXiv  Detail & Related papers  (2025-10-23T14:54:33Z)
• Understanding Catastrophic Interference: On the Identifibility of Latent Representations [67.05452287233122]
  Catastrophic interference, also known as catastrophic forgetting, is a fundamental challenge in machine learning.<n>We propose a novel theoretical framework that formulates catastrophic interference as an identification problem.<n>Our approach provides both theoretical guarantees and practical performance improvements across both synthetic and benchmark datasets.
  arXiv  Detail & Related papers  (2025-09-27T00:53:32Z)
• Tutorial on the Probabilistic Unification of Estimation Theory, Machine Learning, and Generative AI [0.0]
  This survey presents a unified mathematical framework that connects classical estimation theory, statistical inference, and modern machine learning.<n>We show how techniques such as maximum likelihood estimation, Bayesian inference, and attention mechanisms address uncertainty.<n>It serves as both a theoretical synthesis and a practical guide for students and researchers navigating the evolving landscape of machine learning.
  arXiv  Detail & Related papers  (2025-08-21T16:57:33Z)
• Neural Network Reprogrammability: A Unified Theme on Model Reprogramming, Prompt Tuning, and Prompt Instruction [55.914891182214475]
  We introduce neural network reprogrammability as a unifying framework for model adaptation.<n>We present a taxonomy that categorizes such information manipulation approaches across four key dimensions.<n>We also analyze remaining technical challenges and ethical considerations.
  arXiv  Detail & Related papers  (2025-06-05T05:42:27Z)
• Causality can systematically address the monsters under the bench(marks) [64.36592889550431]
  Benchmarks are plagued by various biases, artifacts, or leakage.<n>Models may behave unreliably due to poorly explored failure modes.<n> causality offers an ideal framework to systematically address these challenges.
  arXiv  Detail & Related papers  (2025-02-07T17:01:37Z)
• Failure Modes of LLMs for Causal Reasoning on Narratives [51.19592551510628]
  We investigate the interaction between world knowledge and logical reasoning.<n>We find that state-of-the-art large language models (LLMs) often rely on superficial generalizations.<n>We show that simple reformulations of the task can elicit more robust reasoning behavior.
  arXiv  Detail & Related papers  (2024-10-31T12:48:58Z)
• Causal Abstraction in Model Interpretability: A Compact Survey [5.963324728136442]
  causal abstraction provides a principled approach to understanding and explaining the causal mechanisms underlying model behavior. This survey paper delves into the realm of causal abstraction, examining its theoretical foundations, practical applications, and implications for the field of model interpretability.
  arXiv  Detail & Related papers  (2024-10-26T12:24:28Z)
• A Mechanistic Interpretation of Syllogistic Reasoning in Auto-Regressive Language Models [13.59675117792588]
  Recent studies on logical reasoning in Language Models (LMs) have sparked a debate on whether LMs can learn systematic reasoning principles during pre-training.<n>This paper presents a mechanistic interpretation of syllogistic reasoning in LMs to advance the understanding of internal dynamics.
  arXiv  Detail & Related papers  (2024-08-16T07:47:39Z)
• The Foundations of Tokenization: Statistical and Computational Concerns [51.370165245628975]
  Tokenization is a critical step in the NLP pipeline.<n>Despite its recognized importance as a standard representation method in NLP, the theoretical underpinnings of tokenization are not yet fully understood.<n>The present paper contributes to addressing this theoretical gap by proposing a unified formal framework for representing and analyzing tokenizer models.
  arXiv  Detail & Related papers  (2024-07-16T11:12:28Z)
• Co-designing heterogeneous models: a distributed systems approach [0.40964539027092917]
  This paper presents a modelling approach tailored for heterogeneous systems based on three elements. An inferentialist interpretation of what a model is, a distributed systems metaphor and a co-design cycle describe the practical design and construction of the model. We explore the suitability of this method in the context of three different security-oriented models.
  arXiv  Detail & Related papers  (2024-07-10T13:35:38Z)
• Learning Discrete Concepts in Latent Hierarchical Models [73.01229236386148]
  Learning concepts from natural high-dimensional data holds potential in building human-aligned and interpretable machine learning models.<n>We formalize concepts as discrete latent causal variables that are related via a hierarchical causal model.<n>We substantiate our theoretical claims with synthetic data experiments.
  arXiv  Detail & Related papers  (2024-06-01T18:01:03Z)
• Semantic Objective Functions: A distribution-aware method for adding logical constraints in deep learning [4.854297874710511]
  Constrained Learning and Knowledge Distillation techniques have shown promising results. We propose a loss-based method that embeds knowledge-enforces logical constraints into a machine learning model. We evaluate our method on a variety of learning tasks, including classification tasks with logic constraints.
  arXiv  Detail & Related papers  (2024-05-03T19:21:47Z)
• Explainability for Large Language Models: A Survey [59.67574757137078]
  Large language models (LLMs) have demonstrated impressive capabilities in natural language processing. This paper introduces a taxonomy of explainability techniques and provides a structured overview of methods for explaining Transformer-based language models.
  arXiv  Detail & Related papers  (2023-09-02T22:14:26Z)
• Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability [30.76910454663951]
  Causal abstraction provides a theoretical foundation for mechanistic interpretability.<n>Our contributions are generalizing the theory of causal abstraction from mechanism replacement to arbitrary mechanism transformation.
  arXiv  Detail & Related papers  (2023-01-11T20:42:41Z)
• Developing Constrained Neural Units Over Time [81.19349325749037]
  This paper focuses on an alternative way of defining Neural Networks, that is different from the majority of existing approaches. The structure of the neural architecture is defined by means of a special class of constraints that are extended also to the interaction with data. The proposed theory is cast into the time domain, in which data are presented to the network in an ordered manner.
  arXiv  Detail & Related papers  (2020-09-01T09:07:25Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.