On Catastrophic Inheritance of Large Foundation Models

• URL: http://arxiv.org/abs/2402.01909v2
• Date: Wed, 23 Oct 2024 00:40:23 GMT
• Title: On Catastrophic Inheritance of Large Foundation Models
• Authors: Hao Chen, Bhiksha Raj, Xing Xie, Jindong Wang,
• Abstract summary: Large foundation models (LFMs) are claiming incredible performances. Yet great concerns have been raised about their mythic and uninterpreted potentials. We propose to identify a neglected issue deeply rooted in LFMs: Catastrophic Inheritance. We discuss the challenges behind this issue and propose UIM, a framework to understand the catastrophic inheritance of LFMs from both pre-training and downstream adaptation.
• Score: 51.41727422011327
• License:
• Abstract: Large foundation models (LFMs) are claiming incredible performances. Yet great concerns have been raised about their mythic and uninterpreted potentials not only in machine learning, but also in various other disciplines. In this position paper, we propose to identify a neglected issue deeply rooted in LFMs: Catastrophic Inheritance, describing the weaknesses and limitations inherited from biased large-scale pre-training data to behaviors of LFMs on the downstream tasks, including samples that are corrupted, long-tailed, noisy, out-of-distributed, to name a few. Such inheritance can potentially cause catastrophes to downstream applications, such as bias, lack of generalization, deteriorated performance, security vulnerability, privacy leakage, and value misalignment. We discuss the challenges behind this issue and propose UIM, a framework to Understand the catastrophic inheritance of LFMs from both pre-training and downstream adaptation, Interpret the implications of catastrophic inheritance on downstream tasks, and how to Mitigate it. UIM aims to unite both the machine learning and social sciences communities for more responsible and promising AI development and deployment.

Related papers

• Stars, Stripes, and Silicon: Unravelling the ChatGPT's All-American, Monochrome, Cis-centric Bias [0.0]
  The paper calls for interdisciplinary efforts to address these challenges. It highlights the need for collaboration between researchers, practitioners, and stakeholders to establish governance frameworks.
  arXiv  Detail & Related papers  (2024-10-02T08:55:00Z)
• Metacognitive Myopia in Large Language Models [0.0]
  Large Language Models (LLMs) exhibit potentially harmful biases that reinforce culturally inherent stereotypes, cloud moral judgments, or amplify positive evaluations of majority groups. We propose metacognitive myopia as a cognitive-ecological framework that can account for a conglomerate of established and emerging LLM biases. Our theoretical framework posits that a lack of the two components of metacognition, monitoring and control, causes five symptoms of metacognitive myopia in LLMs.
  arXiv  Detail & Related papers  (2024-08-10T14:43:57Z)
• Detecting and Understanding Vulnerabilities in Language Models via Mechanistic Interpretability [44.99833362998488]
  Large Language Models (LLMs) have shown impressive performance across a wide range of tasks. LLMs in particular are known to be vulnerable to adversarial attacks, where an imperceptible change to the input can mislead the output of the model. We propose a method, based on Mechanistic Interpretability (MI) techniques, to guide this process.
  arXiv  Detail & Related papers  (2024-07-29T09:55:34Z)
• The Reasoning Under Uncertainty Trap: A Structural AI Risk [0.0]
  Report provides an exposition of what makes RUU so challenging for both humans and machines. We detail how this misuse risk connects to a wider network of underlying structural risks.
  arXiv  Detail & Related papers  (2024-01-29T17:16:57Z)
• Model Stealing Attack against Graph Classification with Authenticity, Uncertainty and Diversity [80.16488817177182]
  GNNs are vulnerable to the model stealing attack, a nefarious endeavor geared towards duplicating the target model via query permissions. We introduce three model stealing attacks to adapt to different actual scenarios.
  arXiv  Detail & Related papers  (2023-12-18T05:42:31Z)
• Mitigating Prior Errors in Causal Structure Learning: Towards LLM driven Prior Knowledge [17.634793921251777]
  We aim to tackle erroneous prior causal statements from Large Language Models (LLM) As a pioneer attempt, we propose a BN learning strategy resilient to prior errors without need of human intervention. Specifically, we highlight its substantial ability to resist order-reversed errors while maintaining the majority of correct prior knowledge.
  arXiv  Detail & Related papers  (2023-06-12T11:24:48Z)
• Enhancing Multiple Reliability Measures via Nuisance-extended Information Bottleneck [77.37409441129995]
  In practical scenarios where training data is limited, many predictive signals in the data can be rather from some biases in data acquisition. We consider an adversarial threat model under a mutual information constraint to cover a wider class of perturbations in training. We propose an autoencoder-based training to implement the objective, as well as practical encoder designs to facilitate the proposed hybrid discriminative-generative training.
  arXiv  Detail & Related papers  (2023-03-24T16:03:21Z)
• RelaxLoss: Defending Membership Inference Attacks without Losing Utility [68.48117818874155]
  We propose a novel training framework based on a relaxed loss with a more achievable learning target. RelaxLoss is applicable to any classification model with added benefits of easy implementation and negligible overhead. Our approach consistently outperforms state-of-the-art defense mechanisms in terms of resilience against MIAs.
  arXiv  Detail & Related papers  (2022-07-12T19:34:47Z)
• On the Opportunities and Risks of Foundation Models [256.61956234436553]
  We call these models foundation models to underscore their critically central yet incomplete character. This report provides a thorough account of the opportunities and risks of foundation models. To tackle these questions, we believe much of the critical research on foundation models will require deep interdisciplinary collaboration.
  arXiv  Detail & Related papers  (2021-08-16T17:50:08Z)
• Supercharging Imbalanced Data Learning With Energy-based Contrastive Representation Transfer [72.5190560787569]
  In computer vision, learning from long tailed datasets is a recurring theme, especially for natural image datasets. Our proposal posits a meta-distributional scenario, where the data generating mechanism is invariant across the label-conditional feature distributions. This allows us to leverage a causal data inflation procedure to enlarge the representation of minority classes.
  arXiv  Detail & Related papers  (2020-11-25T00:13:11Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.