Transferable and Efficient Non-Factual Content Detection via Probe Training with Offline Consistency Checking

• URL: http://arxiv.org/abs/2404.06742v1
• Date: Wed, 10 Apr 2024 05:00:35 GMT
• Title: Transferable and Efficient Non-Factual Content Detection via Probe Training with Offline Consistency Checking
• Authors: Xiaokang Zhang, Zijun Yao, Jing Zhang, Kaifeng Yun, Jifan Yu, Juanzi Li, Jie Tang,
• Abstract summary: PINOSE trains a probing model on offline self-consistency checking results, thereby circumventing the need for human-annotated data. It examines various aspects of internal states prior to response decoding, contributing to more effective detection of factual inaccuracies.
• Score: 48.68044413117397
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Detecting non-factual content is a longstanding goal to increase the trustworthiness of large language models (LLMs) generations. Current factuality probes, trained using humanannotated labels, exhibit limited transferability to out-of-distribution content, while online selfconsistency checking imposes extensive computation burden due to the necessity of generating multiple outputs. This paper proposes PINOSE, which trains a probing model on offline self-consistency checking results, thereby circumventing the need for human-annotated data and achieving transferability across diverse data distributions. As the consistency check process is offline, PINOSE reduces the computational burden of generating multiple responses by online consistency verification. Additionally, it examines various aspects of internal states prior to response decoding, contributing to more effective detection of factual inaccuracies. Experiment results on both factuality detection and question answering benchmarks show that PINOSE achieves surpassing results than existing factuality detection methods. Our code and datasets are publicly available on this anonymized repository.

Related papers

• What If the Input is Expanded in OOD Detection? [77.37433624869857]
  Out-of-distribution (OOD) detection aims to identify OOD inputs from unknown classes. Various scoring functions are proposed to distinguish it from in-distribution (ID) data. We introduce a novel perspective, i.e., employing different common corruptions on the input space.
  arXiv  Detail & Related papers  (2024-10-24T06:47:28Z)
• Fact Checking Beyond Training Set [64.88575826304024]
  We show that the retriever-reader suffers from performance deterioration when it is trained on labeled data from one domain and used in another domain. We propose an adversarial algorithm to make the retriever component robust against distribution shift. We then construct eight fact checking scenarios from these datasets, and compare our model to a set of strong baseline models.
  arXiv  Detail & Related papers  (2024-03-27T15:15:14Z)
• Sequential Kernelized Independence Testing [101.22966794822084]
  We design sequential kernelized independence tests inspired by kernelized dependence measures. We demonstrate the power of our approaches on both simulated and real data.
  arXiv  Detail & Related papers  (2022-12-14T18:08:42Z)
• Automatic Change-Point Detection in Time Series via Deep Learning [8.43086628139493]
  We show how to automatically generate new offline detection methods based on training a neural network. We present theory that quantifies the error rate for such an approach, and how it depends on the amount of training data. Our method also shows strong results in detecting and localising changes in activity based on accelerometer data.
  arXiv  Detail & Related papers  (2022-11-07T20:59:14Z)
• Augmenting Softmax Information for Selective Classification with Out-of-Distribution Data [7.221206118679026]
  We show that existing post-hoc methods perform quite differently compared to when evaluated only on OOD detection. We propose a novel method for SCOD, Softmax Information Retaining Combination (SIRC), that augments softmax-based confidence scores with feature-agnostic information. Experiments on a wide variety of ImageNet-scale datasets and convolutional neural network architectures show that SIRC is able to consistently match or outperform the baseline for SCOD.
  arXiv  Detail & Related papers  (2022-07-15T14:39:57Z)
• Adapting to Online Label Shift with Provable Guarantees [137.89382409682233]
  We formulate and investigate the problem of online label shift. The non-stationarity and lack of supervision make the problem challenging to be tackled. Our algorithms enjoy optimal dynamic regret, indicating that performance is competitive with a clairvoyant nature.
  arXiv  Detail & Related papers  (2022-07-05T15:43:14Z)
• Anomaly Detection with Test Time Augmentation and Consistency Evaluation [13.709281244889691]
  We propose a simple, yet effective anomaly detection algorithm named Test Time Augmentation Anomaly Detection (TTA-AD) We observe that in-distribution data enjoy more consistent predictions for its original and augmented versions on a trained network than out-distribution data. Experiments on various high-resolution image benchmark datasets demonstrate that TTA-AD achieves comparable or better detection performance.
  arXiv  Detail & Related papers  (2022-06-06T04:27:06Z)
• Cascade Watchdog: A Multi-tiered Adversarial Guard for Outlier Detection [0.0]
  This paper presents and empirically evaluates a multi-tiered watchdog, which is developed using GAN generated data, for improved out-of-distribution detection. The results show a solid and significant improvement on the detection of the most challenging out-of-distribution inputs while preserving an extremely low false positive rate.
  arXiv  Detail & Related papers  (2021-08-20T21:12:36Z)
• Leveraging Multi-domain, Heterogeneous Data using Deep Multitask Learning for Hate Speech Detection [21.410160004193916]
  We propose a Convolution Neural Network based multi-task learning models (MTLs)footnotecode to leverage information from multiple sources. Empirical analysis performed on three benchmark datasets shows the efficacy of the proposed approach.
  arXiv  Detail & Related papers  (2021-03-23T09:31:01Z)
• Provably Efficient Causal Reinforcement Learning with Confounded Observational Data [135.64775986546505]
  We study how to incorporate the dataset (observational data) collected offline, which is often abundantly available in practice, to improve the sample efficiency in the online setting. We propose the deconfounded optimistic value iteration (DOVI) algorithm, which incorporates the confounded observational data in a provably efficient manner.
  arXiv  Detail & Related papers  (2020-06-22T14:49:33Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.