TRUCE: Private Benchmarking to Prevent Contamination and Improve Comparative Evaluation of LLMs

• URL: http://arxiv.org/abs/2403.00393v2
• Date: Mon, 24 Jun 2024 08:28:18 GMT
• Title: TRUCE: Private Benchmarking to Prevent Contamination and Improve Comparative Evaluation of LLMs
• Authors: Tanmay Rajore, Nishanth Chandran, Sunayana Sitaram, Divya Gupta, Rahul Sharma, Kashish Mittal, Manohar Swaminathan,
• Abstract summary: Benchmarking is the de-facto standard for evaluating LLMs, due to its speed, replicability and low cost. Recent work has pointed out that the majority of the open source benchmarks available today have been contaminated or leaked into LLMs. We propose Private Benchmarking, a solution where test datasets are kept private and models are evaluated without revealing the test data to the model.
• Score: 12.839640915518443
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Benchmarking is the de-facto standard for evaluating LLMs, due to its speed, replicability and low cost. However, recent work has pointed out that the majority of the open source benchmarks available today have been contaminated or leaked into LLMs, meaning that LLMs have access to test data during pretraining and/or fine-tuning. This raises serious concerns about the validity of benchmarking studies conducted so far and the future of evaluation using benchmarks. To solve this problem, we propose Private Benchmarking, a solution where test datasets are kept private and models are evaluated without revealing the test data to the model. We describe various scenarios (depending on the trust placed on model owners or dataset owners), and present solutions to avoid data contamination using private benchmarking. For scenarios where the model weights need to be kept private, we describe solutions from confidential computing and cryptography that can aid in private benchmarking. We build an end-to-end system, TRUCE, that enables such private benchmarking showing that the overheads introduced to protect models and benchmark are negligible (in the case of confidential computing) and tractable (when cryptographic security is required). Finally, we also discuss solutions to the problem of benchmark dataset auditing, to ensure that private benchmarks are of sufficiently high quality.

Related papers

• BenchmarkCards: Large Language Model and Risk Reporting [4.224255134206838]
  Large language models (LLMs) offer powerful capabilities but also introduce significant risks. One way to mitigate these risks is through comprehensive pre-deployment evaluations using benchmarks designed to test for specific vulnerabilities. BenchmarkCards addresses this gap by providing a structured framework specifically for documenting LLM benchmark properties.
  arXiv  Detail & Related papers  (2024-10-16T19:09:02Z)
• Training on the Benchmark Is Not All You Need [52.01920740114261]
  We propose a simple and effective data leakage detection method based on the contents of multiple-choice options. Our method is able to work under black-box conditions without access to model training data or weights. We evaluate the degree of data leakage of 31 mainstream open-source LLMs on four benchmark datasets.
  arXiv  Detail & Related papers  (2024-09-03T11:09:44Z)
• PaCoST: Paired Confidence Significance Testing for Benchmark Contamination Detection in Large Language Models [41.772263447213234]
  Large language models (LLMs) are known to be trained on vast amounts of data, which may unintentionally or intentionally include data from commonly used benchmarks. This inclusion can lead to cheatingly high scores on model leaderboards, yet result in disappointing performance in real-world applications. We introduce PaCoST, a Paired Confidence Significance Testing to effectively detect benchmark contamination in LLMs.
  arXiv  Detail & Related papers  (2024-06-26T13:12:40Z)
• Inference-Time Decontamination: Reusing Leaked Benchmarks for Large Language Model Evaluation [61.350306618479365]
  Leakage of benchmarks can prevent the accurate assessment of large language models' true performance. We propose Inference-Time Decontamination (ITD) to address this issue. ITD reduces inflated accuracy by 22.9% on GSM8K and 19.0% on MMLU.
  arXiv  Detail & Related papers  (2024-06-20T04:35:59Z)
• Privacy-Enhanced Database Synthesis for Benchmark Publishing [16.807486872855534]
  Differential privacy has become a key method for safeguarding privacy when sharing data, but the focus has largely been on minimizing errors in aggregate queries or classification tasks. This paper delves into the creation of privacy-preserving databases specifically for benchmarking, aiming to produce a differentially private database. PrivBench uses sum-product networks (SPNs) to partition and sample data, enhancing data representation while securing privacy.
  arXiv  Detail & Related papers  (2024-05-02T14:20:24Z)
• Benchmarking Benchmark Leakage in Large Language Models [24.015208839742343]
  We introduce a detection pipeline utilizing Perplexity and N-gram accuracy, two simple and scalable metrics that gauge a model's prediction precision on benchmark. We reveal substantial instances of training even test set misuse, resulting in potentially unfair comparisons. We propose the "Benchmark Transparency Card" to encourage clear documentation of benchmark utilization.
  arXiv  Detail & Related papers  (2024-04-29T16:05:36Z)
• Investigating Data Contamination in Modern Benchmarks for Large Language Models [27.479260572913724]
  Recent observations have underscored a disparity between the inflated benchmark scores and the actual performance of LLMs. We study data contamination by proposing two methods tailored for both open-source and proprietary LLMs. We find that certain commercial LLMs could surprisingly guess the missing option in various test sets.
  arXiv  Detail & Related papers  (2023-11-16T11:03:04Z)
• Rethinking Benchmark and Contamination for Language Models with Rephrased Samples [49.18977581962162]
  Large language models are increasingly trained on all the data ever produced by humans. Many have raised concerns about the trustworthiness of public benchmarks due to potential contamination in pre-training or fine-tuning datasets.
  arXiv  Detail & Related papers  (2023-11-08T17:35:20Z)
• Don't Make Your LLM an Evaluation Benchmark Cheater [142.24553056600627]
  Large language models(LLMs) have greatly advanced the frontiers of artificial intelligence, attaining remarkable improvement in model capacity. To assess the model performance, a typical approach is to construct evaluation benchmarks for measuring the ability level of LLMs. We discuss the potential risk and impact of inappropriately using evaluation benchmarks and misleadingly interpreting the evaluation results.
  arXiv  Detail & Related papers  (2023-11-03T14:59:54Z)
• LLMs as Factual Reasoners: Insights from Existing Benchmarks and Beyond [135.8013388183257]
  We propose a new protocol for inconsistency detection benchmark creation and implement it in a 10-domain benchmark called SummEdits. Most LLMs struggle on SummEdits, with performance close to random chance. The best-performing model, GPT-4, is still 8% below estimated human performance.
  arXiv  Detail & Related papers  (2023-05-23T21:50:06Z)
• A Review of Benchmarks for Visual Defect Detection in the Manufacturing Industry [63.52264764099532]
  We propose a study of existing benchmarks to compare and expose their characteristics and their use-cases. A study of industrial metrics requirements, as well as testing procedures, will be presented and applied to the studied benchmarks.
  arXiv  Detail & Related papers  (2023-05-05T07:44:23Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.