Related papers: Do Not Marginalize Mechanisms, Rather Consolidate!

Related papers

On the Importance of Gating: Memorization vs. In-Context Learning in State Space Models [50.824624057327725]
State Space Models (SSMs) have emerged as a compelling alternative to Transformers.<n>We show through theory and experiments that this gating mechanism causes SSMs to first learn an in-weights "memorization" solution.<n>On the other hand, we find that gating is often beneficial for improving generalization to long sequence lengths.
arXiv Detail & Related papers (2026-09-15T02:36:13Z)
CDFM: Towards a General-Purpose Causal Discovery Foundation Model [62.32138962508216]
Causal discovery is the process of recovering underlying causal structures from observational data.<n>We formulate the Causal Discovery Foundation Model (CDFM) as a unified, general-purpose framework for zero-shot structural inference.<n>CDFM consistently outperforms traditional algorithms, driving a paradigm shift toward a general-purpose causal discovery foundation model.
arXiv Detail & Related papers (2026-07-13T12:58:28Z)
CauSim: Scaling Causal Reasoning with Increasingly Complex Causal Simulators [50.047742814033654]
CauSim is a framework that turns causal reasoning from a scarce-label problem into a scalable supervised one.<n>CauSim operates across representations by formalizing non-executable causal knowledge into code, enabling data augmentation, and translating executable SCMs into natural language.
arXiv Detail & Related papers (2026-05-09T17:39:26Z)
Reward Hacking in the Era of Large Models: Mechanisms, Emergent Misalignment, Challenges [87.04241991512386]
We propose the Proxy Compression Hypothesis (PCH) as a unifying framework for understanding reward hacking.<n>Under this view, reward hacking arises from the interaction of objective compression, optimization amplification, and evaluator--policy co-adaptation.<n>This perspective unifies empirical phenomena across RLHF, RLAIF, and RLVR regimes, and explains how local shortcut learning can generalize into broader forms of misalignment.
arXiv Detail & Related papers (2026-04-15T08:11:34Z)
Do Agent Societies Develop Intellectual Elites? The Hidden Power Laws of Collective Cognition in LLM Multi-Agent Systems [5.867974734829712]
We present the first large-scale empirical study of coordination dynamics in multi-agent systems.<n>We analyze over 1.5 Million interactions across tasks, topologies, and scales.<n>We uncover three coupled laws: coordination follows heavy-tailed cascades, concentrates via preferential attachment into intellectual elites, and produces increasingly frequent extreme events as system size grows.
arXiv Detail & Related papers (2026-04-03T03:08:07Z)
Towards a Science of Collective AI: LLM-based Multi-Agent Systems Need a Transition from Blind Trial-and-Error to Rigorous Science [70.3658845234978]
Large Language Models (LLMs) have greatly extended the capabilities of Multi-Agent Systems (MAS)<n>Despite this rapid progress, the field still relies heavily on empirical trial-and-error.<n>This bottleneck stems from the ambiguity of attribution.<n>We propose a factor attribution paradigm to systematically identify collaboration-driving factors.
arXiv Detail & Related papers (2026-02-05T04:19:52Z)
Beyond Quantity: Trajectory Diversity Scaling for Code Agents [51.71414642763219]
Trajectory Diversity Scaling is a data synthesis framework for code agents that scales performance through diversity rather than raw volume.<n> TDScaling integrates four innovations: (1) a Business Cluster mechanism that captures real-service logical dependencies; (2) a blueprint-driven multi-agent paradigm that enforces trajectory coherence; and (3) an adaptive evolution mechanism that steers toward long-tail scenarios.
arXiv Detail & Related papers (2026-02-03T07:43:03Z)
CCMamba: Selective State-Space Models for Higher-Order Graph Learning on Combinatorial Complexes [16.627877999057436]
Topological deep learning has emerged for modeling higher-order structures beyond pairwise interactions.<n>We propose Combinatorial Complex Mamba, the first unified mamba-based neural framework for learning on relational complexes.<n> CCMamba reformulates message passing as a selective state-space modeling problem by organizing multi-rank incidence relations into structured sequences processed by rank-aware state-space models.
arXiv Detail & Related papers (2026-01-28T11:52:13Z)
Computational Thinking Reasoning in Large Language Models [69.28428524878885]
Computational Thinking Model (CTM) is a novel framework that incorporates computational thinking paradigms into large language models (LLMs)<n>Live code execution is seamlessly integrated into the reasoning process, allowing CTM to think by computing.<n>CTM outperforms conventional reasoning models and tool-augmented baselines in terms of accuracy, interpretability, and generalizability.
arXiv Detail & Related papers (2025-06-03T09:11:15Z)
Multi-Agent Collaboration via Evolving Orchestration [61.93162413517026]
Large language models (LLMs) have achieved remarkable results across diverse downstream tasks, but their monolithic nature restricts scalability and efficiency in complex problem-solving.<n>We propose a puppeteer-style paradigm for LLM-based multi-agent collaboration, where a central orchestrator dynamically directs agents in response to evolving task states.<n> Experiments on closed- and open-domain scenarios show that this method achieves superior performance with reduced computational costs.
arXiv Detail & Related papers (2025-05-26T07:02:17Z)
Compression, Regularity, Randomness and Emergent Structure: Rethinking Physical Complexity in the Data-Driven Era [0.0]
We present a unified framework that locates statistical, algorithmic, and dynamical measures along three axes (regularity, randomness, and complexity)<n>This taxonomy reveals the deep challenges posed by uncomputability and highlights the emergence of modern data-driven methods.<n>We close by outlining implications for physics-informed AI and AI-guided discovery in complex physical systems.
arXiv Detail & Related papers (2025-05-12T04:30:42Z)
Syzygy of Thoughts: Improving LLM CoT with the Minimal Free Resolution [59.39066657300045]
Chain-of-Thought (CoT) prompting enhances the reasoning of large language models (LLMs) by decomposing problems into sequential steps. We propose Syzygy of Thoughts (SoT)-a novel framework that extends CoT by introducing auxiliary, interrelated reasoning paths. SoT captures deeper logical dependencies, enabling more robust and structured problem-solving.
arXiv Detail & Related papers (2025-04-13T13:35:41Z)
MCTS-RAG: Enhancing Retrieval-Augmented Generation with Monte Carlo Tree Search [61.11836311160951]
We introduce MCTS-RAG, a novel approach that enhances the reasoning capabilities of small language models on knowledge-intensive tasks.<n>Unlike standard RAG methods, which typically retrieve information independently from reasoning, MCTS-RAG combines structured reasoning with adaptive retrieval.<n>This integrated approach enhances decision-making, reduces hallucinations, and ensures improved factual accuracy and response consistency.
arXiv Detail & Related papers (2025-03-26T17:46:08Z)
Causal Mean Field Multi-Agent Reinforcement Learning [10.767740092703777]
A framework named mean-field reinforcement learning (MFRL) could alleviate the scalability problem by employing the Mean Field Theory. This framework lacks the ability to identify essential interactions under nonstationary environments. We propose an algorithm called causal mean-field Q-learning (CMFQ) to address the scalability problem.
arXiv Detail & Related papers (2025-02-20T02:15:58Z)
Bridging Visualization and Optimization: Multimodal Large Language Models on Graph-Structured Combinatorial Optimization [56.17811386955609]
Graph-structured challenges are inherently difficult due to their nonlinear and intricate nature. In this study, we propose transforming graphs into images to preserve their higher-order structural features accurately. By combining the innovative paradigm powered by multimodal large language models with simple search techniques, we aim to develop a novel and effective framework.
arXiv Detail & Related papers (2025-01-21T08:28:10Z)
Failure Modes of LLMs for Causal Reasoning on Narratives [51.19592551510628]
We investigate the interaction between world knowledge and logical reasoning.<n>We find that state-of-the-art large language models (LLMs) often rely on superficial generalizations.<n>We show that simple reformulations of the task can elicit more robust reasoning behavior.
arXiv Detail & Related papers (2024-10-31T12:48:58Z)
Provable Benefits of Complex Parameterizations for Structured State Space Models [51.90574950170374]
Structured state space models (SSMs) are linear dynamical systems adhering to a specified structure. In contrast to typical neural network modules, whose parameterizations are real, SSMs often use complex parameterizations. This paper takes a step towards explaining the benefits of complex parameterizations for SSMs by establishing formal gaps between real and complex diagonal SSMs.
arXiv Detail & Related papers (2024-10-17T22:35:50Z)
Graph Neural Ordinary Differential Equations for Coarse-Grained Socioeconomic Dynamics [0.0]
We present a data-driven machine-learning approach for modeling space-time socioeconomic dynamics. Our findings, from a case study of Baltimore, MD, indicate that this machine learning-augmented coarse-grained model serves as a powerful instrument.
arXiv Detail & Related papers (2024-07-25T15:12:46Z)
Standardizing Structural Causal Models [80.21199731817698]
We propose internally-standardized structural causal models (iSCMs) for benchmarking algorithms. By construction, iSCMs are not $operatornameVar$-sortable, and as we show experimentally, not $operatornameR2$-sortable either for commonly-used graph families.
arXiv Detail & Related papers (2024-06-17T14:52:21Z)
Can Large Language Models Learn Independent Causal Mechanisms? [9.274428418715347]
Large Language Models (LLMs) fall short on the same tasks in uncommon settings or with distribution shifts. We show that causal models, that learn abstract variables and causal relationships, can demonstrate increased robustness against changes in the distribution.
arXiv Detail & Related papers (2024-02-04T23:04:02Z)
Targeted Reduction of Causal Models [55.11778726095353]
Causal Representation Learning offers a promising avenue to uncover interpretable causal patterns in simulations. We introduce Targeted Causal Reduction (TCR), a method for condensing complex intervenable models into a concise set of causal factors. Its ability to generate interpretable high-level explanations from complex models is demonstrated on toy and mechanical systems.
arXiv Detail & Related papers (2023-11-30T15:46:22Z)
iSCAN: Identifying Causal Mechanism Shifts among Nonlinear Additive Noise Models [48.33685559041322]
This paper focuses on identifying the causal mechanism shifts in two or more related datasets over the same set of variables. Code implementing the proposed method is open-source and publicly available at https://github.com/kevinsbello/iSCAN.
arXiv Detail & Related papers (2023-06-30T01:48:11Z)
Causal Coupled Mechanisms: A Control Method with Cooperation and Competition for Complex System [4.829957501538467]
A novel control method, Causal Coupled Mechanisms, is proposed that explores the cooperation in division and competition in combination. On both synthetic systems and a real-world biological regulatory system, the CCM method achieves robust and state-of-the-art control results even with unpredictable random noise.
arXiv Detail & Related papers (2022-09-15T15:32:16Z)
Causal Inference Through the Structural Causal Marginal Problem [17.91174054672512]
We introduce an approach to counterfactual inference based on merging information from multiple datasets. We formalise this approach for categorical SCMs using the response function formulation and show that it reduces the space of allowed marginal and joint SCMs.
arXiv Detail & Related papers (2022-02-02T21:45:10Z)
Transformers with Competitive Ensembles of Independent Mechanisms [97.93090139318294]
We propose a new Transformer layer which divides the hidden representation and parameters into multiple mechanisms, which only exchange information through attention. We study TIM on a large-scale BERT model, on the Image Transformer, and on speech enhancement and find evidence for semantically meaningful specialization as well as improved performance.
arXiv Detail & Related papers (2021-02-27T21:48:46Z)
Revealing the Invisible with Model and Data Shrinking for Composite-database Micro-expression Recognition [49.463864096615254]
We analyze the influence of learning complexity, including the input complexity and model complexity. We propose a recurrent convolutional network (RCN) to explore the shallower-architecture and lower-resolution input data. We develop three parameter-free modules to integrate with RCN without increasing any learnable parameters.
arXiv Detail & Related papers (2020-06-17T06:19:24Z)
Learning Multivariate Hawkes Processes at Scale [17.17906360554892]
We show that our approach allows to compute the exact likelihood and gradients of an MHP -- independently of the ambient dimensions of the underlying network. We show on synthetic and real-world datasets that our model does not only achieve state-of-the-art predictive results, but also improves runtime performance by multiple orders of magnitude.
arXiv Detail & Related papers (2020-02-28T01:18:01Z)

This list is automatically generated from the titles and abstracts of the papers in this site.