Generalization of LiNGAM that allows confounding

• URL: http://arxiv.org/abs/2401.16661v3
• Date: Thu, 8 Feb 2024 10:13:19 GMT
• Title: Generalization of LiNGAM that allows confounding
• Abstract summary: LiNGAM determines the variable order from cause to effect using additive noise models. Previous methods required significant computational resources regardless of the presence of confounding. This paper enhances LiNGAM by introducing LiNGAM-MMI, a method that quantifies the magnitude of confounding.
• Score: 1.0878040851637998
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: LiNGAM determines the variable order from cause to effect using additive noise models, but it faces challenges with confounding. Previous methods maintained LiNGAM's fundamental structure while trying to identify and address variables affected by confounding. As a result, these methods required significant computational resources regardless of the presence of confounding, and they did not ensure the detection of all confounding types. In contrast, this paper enhances LiNGAM by introducing LiNGAM-MMI, a method that quantifies the magnitude of confounding using KL divergence and arranges the variables to minimize its impact. This method efficiently achieves a globally optimal variable order through the shortest path problem formulation. LiNGAM-MMI processes data as efficiently as traditional LiNGAM in scenarios without confounding while effectively addressing confounding situations. Our experimental results suggest that LiNGAM-MMI more accurately determines the correct variable order, both in the presence and absence of confounding.

Related papers

• From Inference to Adaptation: A Unified Optimal Transport View of Vision Language Model [81.67708143891319]
  Vision-language models (VLMs) have demonstrated remarkable zero-shot capabilities yet remain sensitive to real-world distribution shifts during inference.<n>We bridge the detached objectives of inference and adaptation forVLMs, and propose a principled VLM TTA method called algname.<n>Experiments demonstrate the effectiveness and efficiency of our methods, outperforming the best-performing methods by up to 7% with state-of-the-art efficiency.
  arXiv  Detail & Related papers  (2026-08-18T21:44:32Z)
• Fine-tuning MLLMs Without Forgetting Is Easier Than You Think [72.59321247529975]
  We design a 2x2 experimental framework to assess model performance across in-distribution and out-of-distribution image and text inputs.<n>Our results show that appropriate regularization, such as constraining the number of trainable parameters or adopting a low learning rate, effectively prevents forgetting when dealing with out-of-distribution images.<n>We attribute this forgetting as task-specific overfitting and address this issue by introducing a data-hybrid training strategy.
  arXiv  Detail & Related papers  (2026-03-15T17:16:19Z)
• Task-Awareness Improves LLM Generations and Uncertainty [48.857040212979484]
  Bayes-optimal responses consistently outperform standard decoding methods like beam search.<n>Our decision-theoretic framework is applicable to any problem that admits a latent response structure.
  arXiv  Detail & Related papers  (2026-01-29T10:16:23Z)
• BOTS: A Unified Framework for Bayesian Online Task Selection in LLM Reinforcement Finetuning [82.925106913459]
  Reinforcement finetuning (RFT) is a key technique for aligning Large Language Models (LLMs) with human preferences and enhancing reasoning.<n>We introduce BOTS, a unified framework for Bayesian Online Task Selection in RFT reinforcement finetuning.
  arXiv  Detail & Related papers  (2025-10-30T11:15:23Z)
• In-Context Learning Is Provably Bayesian Inference: A Generalization Theory for Meta-Learning [51.56484100374058]
  We introduce a principled risk decomposition that separates the total ICL risk into two components: Bayes Gap and Posterior Variance.<n>For a uniform-attention Transformer, we derive a non-asymptotic upper bound on this gap, which explicitly clarifies the dependence on the number of pretraining prompts.<n>The Posterior Variance is a model-independent risk representing the intrinsic task uncertainty.
  arXiv  Detail & Related papers  (2025-10-13T03:42:31Z)
• Promptable Anomaly Segmentation with SAM Through Self-Perception Tuning [63.55145330447408]
  Segment Anything Model (SAM) has made great progress in anomaly segmentation tasks due to its impressive generalization ability. Existing methods that directly apply SAM through prompting often overlook the domain shift issue. We propose a novel Self-Perceptinon Tuning (SPT) method, aiming to enhance SAM's perception capability for anomaly segmentation.
  arXiv  Detail & Related papers  (2024-11-26T08:33:25Z)
• Overcoming the Curse of Dimensionality in Reinforcement Learning Through Approximate Factorization [15.898378661128334]
  Reinforcement Learning (RL) algorithms are known to suffer from the curse of dimensionality. We propose overcoming the curse of dimensionality by approximately factorizing the original Markov decision processes (MDPs) into smaller, independently evolving MDPs. We provide improved sample complexity guarantees for both proposed algorithms.
  arXiv  Detail & Related papers  (2024-11-12T07:08:00Z)
• Disentangled Noisy Correspondence Learning [56.06801962154915]
  Cross-modal retrieval is crucial in understanding latent correspondences across modalities. DisNCL is a novel information-theoretic framework for feature Disentanglement in Noisy Correspondence Learning.
  arXiv  Detail & Related papers  (2024-08-10T09:49:55Z)
• Order-Independence Without Fine Tuning [18.020492646988746]
  We present Set-Based Prompting, a technique that guarantees the output of an LLM will not have order dependence on a specified set of sub-sequences.<n>Despite our inputs being out of distribution, the impact on expected accuracy is small, where the expectation is over the order of uniformly chosen shuffling of the candidate responses.
  arXiv  Detail & Related papers  (2024-06-04T16:09:13Z)
• Redefining the Shortest Path Problem Formulation of the Linear Non-Gaussian Acyclic Model: Pairwise Likelihood Ratios, Prior Knowledge, and Path Enumeration [0.0]
  The paper proposes a threefold enhancement to the LiNGAM-SPP framework. The need for parameter tuning is eliminated by using the pairwise likelihood ratio in lieu of kNN-based mutual information. The incorporation of prior knowledge is then enabled by a node-skipping strategy implemented on the graph representation of all causal orderings.
  arXiv  Detail & Related papers  (2024-04-18T05:59:28Z)
• AcceleratedLiNGAM: Learning Causal DAGs at the speed of GPUs [57.12929098407975]
  We show that by efficiently parallelizing existing causal discovery methods, we can scale them to thousands of dimensions. Specifically, we focus on the causal ordering subprocedure in DirectLiNGAM and implement GPU kernels to accelerate it. This allows us to apply DirectLiNGAM to causal inference on large-scale gene expression data with genetic interventions yielding competitive results.
  arXiv  Detail & Related papers  (2024-03-06T15:06:11Z)
• Merging Multi-Task Models via Weight-Ensembling Mixture of Experts [64.94129594112557]
  Merging Transformer-based models trained on different tasks into a single unified model can execute all the tasks concurrently. Previous methods, exemplified by task arithmetic, have been proven to be both effective and scalable. We propose to merge most of the parameters while upscaling the Transformer layers to a weight-ensembling mixture of experts (MoE) module.
  arXiv  Detail & Related papers  (2024-02-01T08:58:57Z)
• Winning Prize Comes from Losing Tickets: Improve Invariant Learning by Exploring Variant Parameters for Out-of-Distribution Generalization [76.27711056914168]
  Out-of-Distribution (OOD) Generalization aims to learn robust models that generalize well to various environments without fitting to distribution-specific features. Recent studies based on Lottery Ticket Hypothesis (LTH) address this problem by minimizing the learning target to find some of the parameters that are critical to the task. We propose Exploring Variant parameters for Invariant Learning (EVIL) which also leverages the distribution knowledge to find the parameters that are sensitive to distribution shift.
  arXiv  Detail & Related papers  (2023-10-25T06:10:57Z)
• Amortizing intractable inference in large language models [56.92471123778389]
  We use amortized Bayesian inference to sample from intractable posterior distributions. We empirically demonstrate that this distribution-matching paradigm of LLM fine-tuning can serve as an effective alternative to maximum-likelihood training. As an important application, we interpret chain-of-thought reasoning as a latent variable modeling problem.
  arXiv  Detail & Related papers  (2023-10-06T16:36:08Z)
• TSLiNGAM: DirectLiNGAM under heavy tails [0.0]
  We propose TSLiNGAM, a new method for identifying the DAG of a causal model based on observational data. TSLiNGAM builds on DirectLiNGAM, a popular algorithm which uses simple OLS regression for identifying causal directions between variables. It performs significantly better on heavy-tailed and skewed data and demonstrates a high small-sample efficiency.
  arXiv  Detail & Related papers  (2023-08-10T08:34:46Z)
• Relational Reasoning via Set Transformers: Provable Efficiency and Applications to MARL [154.13105285663656]
  A cooperative Multi-A gent R einforcement Learning (MARL) with permutation invariant agents framework has achieved tremendous empirical successes in real-world applications. Unfortunately, the theoretical understanding of this MARL problem is lacking due to the curse of many agents and the limited exploration of the relational reasoning in existing works. We prove that the suboptimality gaps of the model-free and model-based algorithms are independent of and logarithmic in the number of agents respectively, which mitigates the curse of many agents.
  arXiv  Detail & Related papers  (2022-09-20T16:42:59Z)
• Causal Order Identification to Address Confounding: Binary Variables [4.56877715768796]
  This paper considers an extension of the linear non-Gaussian acyclic model (LiNGAM) LiNGAM determines the causal order among variables from a dataset when the variables are expressed by a set of linear equations, including noise.
  arXiv  Detail & Related papers  (2021-08-10T22:09:43Z)
• Generalized ADMM in Distributed Learning via Variational Inequality [7.013072937051018]
  Alternating Direction Method of Multipliers (ADMM) through the concept of consensus variables is a practical algorithm. In this paper, we study the effect of the local data sets of users in the distributed learning of ADMM.
  arXiv  Detail & Related papers  (2021-04-26T14:16:14Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.