On the Universal Truthfulness Hyperplane Inside LLMs

• URL: http://arxiv.org/abs/2407.08582v1
• Date: Thu, 11 Jul 2024 15:07:26 GMT
• Title: On the Universal Truthfulness Hyperplane Inside LLMs
• Authors: Junteng Liu, Shiqi Chen, Yu Cheng, Junxian He,
• Abstract summary: We investigate whether a universal truthfulness hyperplane that distinguishes the model's factually correct and incorrect outputs exists within the model. Our results indicate that increasing the diversity of the training datasets significantly enhances the performance in all scenarios.
• Score: 27.007142483859162
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While large language models (LLMs) have demonstrated remarkable abilities across various fields, hallucination remains a significant challenge. Recent studies have explored hallucinations through the lens of internal representations, proposing mechanisms to decipher LLMs' adherence to facts. However, these approaches often fail to generalize to out-of-distribution data, leading to concerns about whether internal representation patterns reflect fundamental factual awareness, or only overfit spurious correlations on the specific datasets. In this work, we investigate whether a universal truthfulness hyperplane that distinguishes the model's factually correct and incorrect outputs exists within the model. To this end, we scale up the number of training datasets and conduct an extensive evaluation -- we train the truthfulness hyperplane on a diverse collection of over 40 datasets and examine its cross-task, cross-domain, and in-domain generalization. Our results indicate that increasing the diversity of the training datasets significantly enhances the performance in all scenarios, while the volume of data samples plays a less critical role. This finding supports the optimistic hypothesis that a universal truthfulness hyperplane may indeed exist within the model, offering promising directions for future research.

Related papers

• Learning Divergence Fields for Shift-Robust Graph Representations [73.11818515795761]
  In this work, we propose a geometric diffusion model with learnable divergence fields for the challenging problem with interdependent data. We derive a new learning objective through causal inference, which can guide the model to learn generalizable patterns of interdependence that are insensitive across domains.
  arXiv  Detail & Related papers  (2024-06-07T14:29:21Z)
• Deriving Causal Order from Single-Variable Interventions: Guarantees & Algorithm [14.980926991441345]
  We show that datasets containing interventional data can be effectively extracted under realistic assumptions about the data distribution. We introduce interventional faithfulness, which relies on comparisons between the marginal distributions of each variable across observational and interventional settings. We also introduce Intersort, an algorithm designed to infer the causal order from datasets containing large numbers of single-variable interventions.
  arXiv  Detail & Related papers  (2024-05-28T16:07:17Z)
• Unveiling the Generalization Power of Fine-Tuned Large Language Models [81.70754292058258]
  We investigate whether fine-tuning affects the intrinsic generalization ability intrinsic to Large Language Models (LLMs) Our main findings reveal that models fine-tuned on generation and classification tasks exhibit dissimilar behaviors in generalizing to different domains and tasks. We observe that integrating the in-context learning strategy during fine-tuning on generation tasks can enhance the model's generalization ability.
  arXiv  Detail & Related papers  (2024-03-14T08:18:59Z)
• Machine Learning vs Deep Learning: The Generalization Problem [0.0]
  This study investigates the comparative abilities of traditional machine learning (ML) models and deep learning (DL) algorithms in terms of extrapolation. We present an empirical analysis where both ML and DL models are trained on an exponentially growing function and then tested on values outside the training domain. Our findings suggest that deep learning models possess inherent capabilities to generalize beyond the training scope.
  arXiv  Detail & Related papers  (2024-03-03T21:42:55Z)
• Dive into the Chasm: Probing the Gap between In- and Cross-Topic Generalization [66.4659448305396]
  This study analyzes various LMs with three probing-based experiments to shed light on the reasons behind the In- vs. Cross-Topic generalization gap. We demonstrate, for the first time, that generalization gaps and the robustness of the embedding space vary significantly across LMs.
  arXiv  Detail & Related papers  (2024-02-02T12:59:27Z)
• Leveraging Diffusion Disentangled Representations to Mitigate Shortcuts in Underspecified Visual Tasks [92.32670915472099]
  We propose an ensemble diversification framework exploiting the generation of synthetic counterfactuals using Diffusion Probabilistic Models (DPMs) We show that diffusion-guided diversification can lead models to avert attention from shortcut cues, achieving ensemble diversity performance comparable to previous methods requiring additional data collection.
  arXiv  Detail & Related papers  (2023-10-03T17:37:52Z)
• The Rashomon Importance Distribution: Getting RID of Unstable, Single Model-based Variable Importance [16.641794438414745]
  Quantifying variable importance is essential for answering high-stakes questions in fields like genetics, public policy, and medicine. We propose a new variable importance framework that quantifies the importance of a variable across the set of all good models and is stable across the data distribution. Our framework is extremely flexible and can be integrated with most existing model classes and global variable importance metrics.
  arXiv  Detail & Related papers  (2023-09-24T23:09:48Z)
• Leveraging sparse and shared feature activations for disentangled representation learning [112.22699167017471]
  We propose to leverage knowledge extracted from a diversified set of supervised tasks to learn a common disentangled representation. We validate our approach on six real world distribution shift benchmarks, and different data modalities.
  arXiv  Detail & Related papers  (2023-04-17T01:33:24Z)
• A Comprehensive Review of Trends, Applications and Challenges In Out-of-Distribution Detection [0.76146285961466]
  Field of study has emerged, focusing on detecting out-of-distribution data subsets and enabling a more comprehensive generalization. As many deep learning based models have achieved near-perfect results on benchmark datasets, the need to evaluate these models' reliability and trustworthiness is felt more strongly than ever. This paper presents a survey that, in addition to reviewing more than 70 papers in this field, presents challenges and directions for future works and offers a unifying look into various types of data shifts and solutions for better generalization.
  arXiv  Detail & Related papers  (2022-09-26T18:13:14Z)
• CHALLENGER: Training with Attribution Maps [63.736435657236505]
  We show that utilizing attribution maps for training neural networks can improve regularization of models and thus increase performance. In particular, we show that our generic domain-independent approach yields state-of-the-art results in vision, natural language processing and on time series tasks.
  arXiv  Detail & Related papers  (2022-05-30T13:34:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.