Related papers: Intrinsic Barriers to Explaining Deep Foundation Models

Related papers

From Passive Metric to Active Signal: The Evolving Role of Uncertainty Quantification in Large Language Models [77.04403907729738]
This survey charts the evolution of uncertainty from a passive diagnostic metric to an active control signal guiding real-time model behavior.<n>We demonstrate how uncertainty is leveraged as an active control signal across three frontiers.<n>This survey argues that mastering the new trend of uncertainty is essential for building the next generation of scalable, reliable, and trustworthy AI.
arXiv Detail & Related papers (2026-01-22T06:21:31Z)
When the Coffee Feature Activates on Coffins: An Analysis of Feature Extraction and Steering for Mechanistic Interpretability [0.0]
Recent work by Anthropic on Mechanistic interpretability claims to understand and control Large Language Models.<n>We conduct an initial stress-test of these claims by replicating their main results with open-source SAEs for Llama 3.1.<n>We find that feature steering exhibits substantial fragility, with sensitivity to layer selection, steering magnitude, and context.
arXiv Detail & Related papers (2026-01-06T14:29:51Z)
Beware of Reasoning Overconfidence: Pitfalls in the Reasoning Process for Multi-solution Tasks [54.31998314008198]
Large Language Models (LLMs) excel in reasoning tasks requiring a single correct answer, but they perform poorly in multi-solution tasks.<n>We attribute this limitation to textbfreasoning overconfidence: a tendency to express undue certainty in an incomplete solution set.<n>We propose the textbfcognitive-rigidity hypothesis, which posits that overconfidence arises when the reasoning process prematurely converges on a narrow set of thought paths.
arXiv Detail & Related papers (2025-12-01T14:35:06Z)
From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Models [66.36007274540113]
Multimodal Large Language Models (MLLMs) strive to achieve a profound, human-like understanding of and interaction with the physical world.<n>They often exhibit a shallow and incoherent integration when acquiring information (Perception) and conducting reasoning (Cognition)<n>This survey introduces a novel and unified analytical framework: From Perception to Cognition"
arXiv Detail & Related papers (2025-09-29T18:25:40Z)
Responsible Diffusion: A Comprehensive Survey on Safety, Ethics, and Trust in Diffusion Models [69.22690439422531]
Diffusion models (DMs) have been investigated in various domains due to their ability to generate high-quality data.<n>Similar to traditional deep learning systems, there also exist potential threats to DMs.<n>This survey comprehensively elucidates its framework, threats, and countermeasures.
arXiv Detail & Related papers (2025-09-25T02:51:43Z)
A Comprehensive Survey on the Risks and Limitations of Concept-based Models [33.641361996627175]
Concept-based Models are inherently explainable networks that improve upon standard Deep Neural Networks.<n>These models are highly successful in critical applications like medical diagnosis and financial risk prediction.<n>However, recent research has uncovered significant limitations in the structure of such networks.
arXiv Detail & Related papers (2025-05-25T03:53:26Z)
Robustness in Large Language Models: A Survey of Mitigation Strategies and Evaluation Metrics [0.7481505949203433]
Large Language Models (LLMs) have emerged as a promising cornerstone for the development of natural language processing (NLP) and artificial intelligence (AI)<n>This survey provides a comprehensive overview of current studies in this area.
arXiv Detail & Related papers (2025-05-24T11:50:52Z)
Causality-Driven Neural Network Repair: Challenges and Opportunities [5.69361786082969]
Deep Neural Networks (DNNs) often rely on statistical correlations rather than causal reasoning, limiting their robustness and interpretability. This paper explores causal inference as an approach primarily for DNN repair, leveraging causal debug, and structural causal models (SCMs) to identify and correct failures.
arXiv Detail & Related papers (2025-04-24T21:22:00Z)
All You Need for Counterfactual Explainability Is Principled and Reliable Estimate of Aleatoric and Epistemic Uncertainty [27.344785490275864]
We argue that transparency research overlooks many foundational concepts of artificial intelligence.<n>Inherently transparent models can benefit from human-centred explanatory insights.<n>At a higher level, integrating artificial intelligence fundamentals into transparency research promises to yield more reliable, robust and understandable predictive models.
arXiv Detail & Related papers (2025-02-24T09:38:31Z)
Open Problems in Mechanistic Interpretability [61.44773053835185]
Mechanistic interpretability aims to understand the computational mechanisms underlying neural networks' capabilities.<n>Despite recent progress toward these goals, there are many open problems in the field that require solutions.
arXiv Detail & Related papers (2025-01-27T20:57:18Z)
A Theoretical Survey on Foundation Models [48.2313835471321]
This survey aims to review those interpretable methods that comply with the aforementioned principles and have been successfully applied to black-box foundation models. The methods are deeply rooted in machine learning theory, covering the analysis of generalization performance, expressive capability, and dynamic behavior. They provide a thorough interpretation of the entire workflow of FMs, ranging from the inference capability and training dynamics to their ethical implications.
arXiv Detail & Related papers (2024-10-15T09:48:03Z)
FFAA: Multimodal Large Language Model based Explainable Open-World Face Forgery Analysis Assistant [59.2438504610849]
We introduce FFAA: Face Forgery Analysis Assistant, consisting of a fine-tuned Multimodal Large Language Model (MLLM) and Multi-answer Intelligent Decision System (MIDS) Our method not only provides user-friendly and explainable results but also significantly boosts accuracy and robustness compared to previous methods.
arXiv Detail & Related papers (2024-08-19T15:15:20Z)
Learning Structural Causal Models through Deep Generative Models: Methods, Guarantees, and Challenges [42.0626213927983]
It analyzes the hypotheses, guarantees, and applications inherent to the underlying deep learning components and structural causal models. It highlights the challenges and open questions in the field of deep structural causal modeling.
arXiv Detail & Related papers (2024-05-08T12:56:33Z)
On the Challenges and Opportunities in Generative AI [157.96723998647363]
We argue that current large-scale generative AI models exhibit several fundamental shortcomings that hinder their widespread adoption across domains.<n>We aim to provide researchers with insights for exploring fruitful research directions, thus fostering the development of more robust and accessible generative AI solutions.
arXiv Detail & Related papers (2024-02-28T15:19:33Z)
On Catastrophic Inheritance of Large Foundation Models [51.41727422011327]
Large foundation models (LFMs) are claiming incredible performances. Yet great concerns have been raised about their mythic and uninterpreted potentials. We propose to identify a neglected issue deeply rooted in LFMs: Catastrophic Inheritance. We discuss the challenges behind this issue and propose UIM, a framework to understand the catastrophic inheritance of LFMs from both pre-training and downstream adaptation.
arXiv Detail & Related papers (2024-02-02T21:21:55Z)
Towards CausalGPT: A Multi-Agent Approach for Faithful Knowledge Reasoning via Promoting Causal Consistency in LLMs [55.66353783572259]
Causal-Consistency Chain-of-Thought harnesses multi-agent collaboration to bolster the faithfulness and causality of foundation models. Our framework demonstrates significant superiority over state-of-the-art methods through extensive and comprehensive evaluations.
arXiv Detail & Related papers (2023-08-23T04:59:21Z)
Explainable Deep Reinforcement Learning: State of the Art and Challenges [1.005130974691351]
Interpretability, explainability and transparency are key issues to introducing Artificial Intelligence methods in many critical domains. This article provides a review of state of the art methods for explainable deep reinforcement learning methods.
arXiv Detail & Related papers (2023-01-24T11:41:25Z)
Towards a Responsible AI Development Lifecycle: Lessons From Information Security [0.0]
We propose a framework for responsibly developing artificial intelligence systems. In particular, we propose leveraging the concepts of threat modeling, design review, penetration testing, and incident response.
arXiv Detail & Related papers (2022-03-06T13:03:58Z)
Counterfactual Explanations as Interventions in Latent Space [62.997667081978825]
Counterfactual explanations aim to provide to end users a set of features that need to be changed in order to achieve a desired outcome. Current approaches rarely take into account the feasibility of actions needed to achieve the proposed explanations. We present Counterfactual Explanations as Interventions in Latent Space (CEILS), a methodology to generate counterfactual explanations.
arXiv Detail & Related papers (2021-06-14T20:48:48Z)

This list is automatically generated from the titles and abstracts of the papers in this site.