Test Time Training for Supervised Causal Learning

• URL: http://arxiv.org/abs/2605.30015v1
• Date: Thu, 28 May 2026 14:39:49 GMT
• Title: Test Time Training for Supervised Causal Learning
• Abstract summary: Supervised Causal Learning (SCL) has shown promise in causal discovery by framing it as a supervised learning problem.<n>We reveal three limitations of previous SCL practices: a significant performance gap between synthetic benchmarks and real-world data, fragility to distribution shifts, and failure in compositional generalization.<n>We propose Test-Time Training for Supervised Causal Learning (TTT-SCL), a novel framework that dynamically generates training sets explicitly aligned with any specific test instance.
• Score: 30.744868688801684
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Supervised Causal Learning (SCL) has shown promise in causal discovery by framing it as a supervised learning problem. However, it suffers from significant out-of-distribution generalization challenges. We reveal three limitations of previous SCL practices: a significant performance gap between synthetic benchmarks and real-world data, fragility to distribution shifts, and failure in compositional generalization, collectively questioning its real-world applicability. To address this, we propose Test-Time Training for Supervised Causal Learning (TTT-SCL), a novel framework that dynamically generates training sets explicitly aligned with any specific test instance. We demonstrate the correlation between TTT-SCL and score-based methods, and design an efficient module for generating training sets based on the classic scoring function. Experiments on synthetic benchmarks, pseudo-real and real-world datasets demonstrate that TTT-SCL significantly outperforms existing SCL and traditional causal discovery methods.

Related papers

• Rethinking Easy-to-Hard: Limits of Curriculum Learning in Post-Training for Deductive Reasoning [61.81622271655058]
  We present a systematic empirical study of curriculum-based post-training of large language models.<n>We find no robust advantage in difficulty-based sequencing over standard random sampling in either accuracy or response length.<n>Our study suggests that, in the context of deductive reasoning, the specific ordering of training examples plays a negligible role in achieving compositional generalization.
  arXiv  Detail & Related papers  (2026-03-28T10:43:36Z)
• Demonstrations, CoT, and Prompting: A Theoretical Analysis of ICL [9.83462430479849]
  In-Context Learning (ICL) enables pretrained LLMs to adapt to downstream tasks by conditioning on a small set of input-output demonstrations.<n>We analyze how ICL performance is governed by (i) the quality of selected demonstrations, (ii) an intrinsic ICL capability of the pretrained model, and (iii) the degree of distribution shift.
  arXiv  Detail & Related papers  (2026-03-20T03:33:04Z)
• Test-Time Learning of Causal Structure from Interventional Data [50.06913286558919]
  We propose TICL (Test-time Interventional Causal Learning), a novel method that synergizes Test-Time Training with Joint Causal Inference.<n>Specifically, we design a self-augmentation strategy to generate instance-specific training data at test time, effectively avoiding distribution shifts.<n>By integrating joint causal inference, we developed a PC-inspired two-phase supervised learning scheme, which effectively leverages self-augmented training data while ensuring theoretical identifiability.
  arXiv  Detail & Related papers  (2026-02-22T11:23:05Z)
• LibContinual: A Comprehensive Library towards Realistic Continual Learning [62.34449396069085]
  A fundamental challenge in Continual Learning (CL) is catastrophic forgetting, where adapting to new tasks degrades the performance on previous ones.<n>We propose LibContinual, a comprehensive and reproducible library designed to serve as a foundational platform for realistic CL.
  arXiv  Detail & Related papers  (2025-12-26T13:59:13Z)
• The Curse of CoT: On the Limitations of Chain-of-Thought in In-Context Learning [56.574829311863446]
  Chain-of-Thought (CoT) prompting has been widely recognized for its ability to enhance reasoning capabilities in large language models (LLMs)<n>We demonstrate that CoT and its reasoning variants consistently underperform direct answering across varying model scales and benchmark complexities.<n>Our analysis uncovers a fundamental hybrid mechanism of explicit-implicit reasoning driving CoT's performance in pattern-based ICL.
  arXiv  Detail & Related papers  (2025-04-07T13:51:06Z)
• Does the Definition of Difficulty Matter? Scoring Functions and their Role for Curriculum Learning [42.4526628515253]
  Curriculum learning (CL) describes a machine learning training strategy in which samples are gradually introduced into the training process based on their difficulty. We study the robustness and similarity of the most common scoring functions for sample difficulty estimation. We find that the robustness of scoring functions across random seeds positively correlates with CL performance.
  arXiv  Detail & Related papers  (2024-11-01T18:55:31Z)
• LoRanPAC: Low-rank Random Features and Pre-trained Models for Bridging Theory and Practice in Continual Learning [103.45785408116146]
  Continual learning (CL) aims to train a model that can solve multiple tasks presented sequentially.<n>Recent CL approaches have achieved strong performance by leveraging large pre-trained models that generalize well to downstream tasks.<n>However, such methods lack theoretical guarantees, making them prone to unexpected failures.<n>We aim to bridge this gap by designing a simple CL method that is theoretically sound and highly performant.
  arXiv  Detail & Related papers  (2024-10-01T12:58:37Z)
• Investigating the Pre-Training Dynamics of In-Context Learning: Task Recognition vs. Task Learning [99.05401042153214]
  In-context learning (ICL) is potentially attributed to two major abilities: task recognition (TR) and task learning (TL) We take the first step by examining the pre-training dynamics of the emergence of ICL. We propose a simple yet effective method to better integrate these two abilities for ICL at inference time.
  arXiv  Detail & Related papers  (2024-06-20T06:37:47Z)
• Relaxed Contrastive Learning for Federated Learning [48.96253206661268]
  We propose a novel contrastive learning framework to address the challenges of data heterogeneity in federated learning. Our framework outperforms all existing federated learning approaches by huge margins on the standard benchmarks.
  arXiv  Detail & Related papers  (2024-01-10T04:55:24Z)
• Density Distribution-based Learning Framework for Addressing Online Continual Learning Challenges [4.715630709185073]
  We introduce a density distribution-based learning framework for online Continual Learning. Our framework achieves superior average accuracy and time-space efficiency. Our method outperforms popular CL approaches by a significant margin.
  arXiv  Detail & Related papers  (2023-11-22T09:21:28Z)
• Unbiased and Efficient Self-Supervised Incremental Contrastive Learning [31.763904668737304]
  We propose a self-supervised Incremental Contrastive Learning (ICL) framework consisting of a novel Incremental InfoNCE (NCE-II) loss function. ICL achieves up to 16.7x training speedup and 16.8x faster convergence with competitive results.
  arXiv  Detail & Related papers  (2023-01-28T06:11:31Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.