A Dialogue between Causal and Traditional Representation Learning: Toward Mutual Benefits in a Unified Formulation

• URL: http://arxiv.org/abs/2605.21058v1
• Date: Wed, 20 May 2026 11:43:46 GMT
• Title: A Dialogue between Causal and Traditional Representation Learning: Toward Mutual Benefits in a Unified Formulation
• Abstract summary: Causal representation learning and traditional representation learning have largely developed along different trajectories.<n>We argue that these two fields should be brought into dialogue rather than treated as separate paradigms.
• Score: 20.79575403725719
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Causal representation learning (CRL) and traditional representation learning have largely developed along different trajectories. Traditional representation learning has been driven mainly by applications and empirical objectives, whereas CRL has focused more on theoretical questions, particularly identifiability. This difference in emphasis has created a gap between the two fields in terminology, problem formulation, and evaluation, limiting communication and sometimes leading to disconnected or redundant efforts. In this paper, we argue that these two fields should be brought into dialogue rather than treated as separate paradigms. To this end, we introduce a unified formulation in which the representation learning is characterized by two components: a task component, which specifies what information the learned representation is required to preserve, and a constraint component, which specifies what structure is imposed on the latent space. Under this formulation, the benefits run in both directions. CRL provides theoretical tools for understanding when structured latent constraints are useful or necessary, while traditional representation learning offers practical insights on task design and objective choice that can improve the development of CRL methods. To illustrate this interaction, we experimentally study how different task components affect the behavior of CRL methods under different structured constraints. Results on CausalVerse show that the effectiveness of causal constraints depends strongly on the tasks with which they are paired.

Related papers

• An Introduction to Causal Reinforcement Learning [58.680653905480284]
  Causal inference and reinforcement learning operate over different aspects of the same building block, counterfactual relations.<n>We put under a unifying treatment different modes of learning, including online, off-policy, and causal calculus learning.<n>Specifically, we introduce and discuss through causal lenses generalized policy learning, where to intervene, imitation learning, and counterfactual learning.
  arXiv  Detail & Related papers  (2026-06-23T05:28:33Z)
• Understanding Catastrophic Interference: On the Identifibility of Latent Representations [67.05452287233122]
  Catastrophic interference, also known as catastrophic forgetting, is a fundamental challenge in machine learning.<n>We propose a novel theoretical framework that formulates catastrophic interference as an identification problem.<n>Our approach provides both theoretical guarantees and practical performance improvements across both synthetic and benchmark datasets.
  arXiv  Detail & Related papers  (2025-09-27T00:53:32Z)
• Can One Domain Help Others? A Data-Centric Study on Multi-Domain Reasoning via Reinforcement Learning [32.665418383317224]
  We present a systematic investigation of multi-domain reasoning within the RLVR framework.<n>We focus on three primary domains: mathematical reasoning, code generation, and logical puzzle solving.<n>Our results offer significant insights into the dynamics governing domain interactions.
  arXiv  Detail & Related papers  (2025-07-23T13:51:04Z)
• A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems [93.8285345915925]
  Reasoning is a fundamental cognitive process that enables logical inference, problem-solving, and decision-making.<n>With the rapid advancement of large language models (LLMs), reasoning has emerged as a key capability that distinguishes advanced AI systems.<n>We categorize existing methods along two dimensions: (1) Regimes, which define the stage at which reasoning is achieved; and (2) Architectures, which determine the components involved in the reasoning process.
  arXiv  Detail & Related papers  (2025-04-12T01:27:49Z)
• Continual Learning Should Move Beyond Incremental Classification [51.23416308775444]
  Continual learning (CL) is the sub-field of machine learning concerned with accumulating knowledge in dynamic environments.<n>Here, we argue that maintaining such a focus limits both theoretical development and practical applicability of CL methods.<n>We identify three fundamental challenges: (C1) the nature of continuity in learning problems, (C2) the choice of appropriate spaces and metrics for measuring similarity, and (C3) the role of learning objectives beyond classification.
  arXiv  Detail & Related papers  (2025-02-17T15:40:13Z)
• CausalCOMRL: Context-Based Offline Meta-Reinforcement Learning with Causal Representation [13.575628222213387]
  CausalCOMRL is a context-based OMRL method that integrates causal representation learning.<n>We show that CausalCOMRL achieves better performance than other methods on most benchmarks.
  arXiv  Detail & Related papers  (2025-02-03T01:43:54Z)
• Learning by Doing: An Online Causal Reinforcement Learning Framework with Causal-Aware Policy [38.86867078596718]
  We consider explicitly modeling the generation process of states with the graphical causal model.<n>We formulate the causal structure updating into the RL interaction process with active intervention learning of the environment.
  arXiv  Detail & Related papers  (2024-02-07T14:09:34Z)
• Towards an Information Theoretic Framework of Context-Based Offline Meta-Reinforcement Learning [48.79569442193824]
  We show that COMRL algorithms are essentially optimizing the same mutual information objective between the task variable $M$ and its latent representation $Z$ by implementing various approximate bounds.<n>As demonstrations, we propose a supervised and a self-supervised implementation of $I(Z; M)$, and empirically show that the corresponding optimization algorithms exhibit remarkable generalization across a broad spectrum of RL benchmarks.<n>This work lays the information theoretic foundation for COMRL methods, leading to a better understanding of task representation learning in the context of reinforcement learning.
  arXiv  Detail & Related papers  (2024-02-04T09:58:42Z)
• SSL Framework for Causal Inconsistency between Structures and Representations [31.895570222735955]
  Cross-pollination between causal discovery and deep learning has led to increasingly extensive interactions.<n>Indefinite Data has conflicts between causal relationships expressed by the causal structure and causal representation generated by deep learning models.<n>To alleviate causal inconsistency, we proposed a self-supervised learning framework based on intervention.
  arXiv  Detail & Related papers  (2023-10-28T08:29:49Z)
• Conceptual Reinforcement Learning for Language-Conditioned Tasks [20.300727364957208]
  We propose a conceptual reinforcement learning (CRL) framework to learn the concept-like joint representation for language-conditioned policy. The key insight is that concepts are compact and invariant representations in human instances and in real-world situations.
  arXiv  Detail & Related papers  (2023-03-09T07:01:06Z)
• Disentangled Representation Learning [46.51815065323667]
  Disentangled Representation Learning (DRL) aims to learn a model capable of identifying and disentangling the underlying factors hidden in the observable data in representation form. We comprehensively investigate DRL from various aspects including motivations, definitions, methodologies, evaluations, applications, and model designs.
  arXiv  Detail & Related papers  (2022-11-21T18:14:38Z)
• Contextualize Me -- The Case for Context in Reinforcement Learning [49.794253971446416]
  Contextual Reinforcement Learning (cRL) provides a framework to model such changes in a principled manner. We show how cRL contributes to improving zero-shot generalization in RL through meaningful benchmarks and structured reasoning about generalization tasks.
  arXiv  Detail & Related papers  (2022-02-09T15:01:59Z)
• Which Mutual-Information Representation Learning Objectives are Sufficient for Control? [80.2534918595143]
  Mutual information provides an appealing formalism for learning representations of data. This paper formalizes the sufficiency of a state representation for learning and representing the optimal policy. Surprisingly, we find that two of these objectives can yield insufficient representations given mild and common assumptions on the structure of the MDP.
  arXiv  Detail & Related papers  (2021-06-14T10:12:34Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.