Can LLMs be Fooled? Investigating Vulnerabilities in LLMs

• URL: http://arxiv.org/abs/2407.20529v1
• Date: Tue, 30 Jul 2024 04:08:00 GMT
• Title: Can LLMs be Fooled? Investigating Vulnerabilities in LLMs
• Authors: Sara Abdali, Jia He, CJ Barberan, Richard Anarfi,
• Abstract summary: The advent of Large Language Models (LLMs) has garnered significant popularity and wielded immense power across various domains within Natural Language Processing (NLP) This paper will synthesize the findings from each vulnerability section and propose new directions of research and development. By understanding the focal points of current vulnerabilities, we can better anticipate and mitigate future risks.
• Score: 4.927763944523323
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: The advent of Large Language Models (LLMs) has garnered significant popularity and wielded immense power across various domains within Natural Language Processing (NLP). While their capabilities are undeniably impressive, it is crucial to identify and scrutinize their vulnerabilities especially when those vulnerabilities can have costly consequences. One such LLM, trained to provide a concise summarization from medical documents could unequivocally leak personal patient data when prompted surreptitiously. This is just one of many unfortunate examples that have been unveiled and further research is necessary to comprehend the underlying reasons behind such vulnerabilities. In this study, we delve into multiple sections of vulnerabilities which are model-based, training-time, inference-time vulnerabilities, and discuss mitigation strategies including "Model Editing" which aims at modifying LLMs behavior, and "Chroma Teaming" which incorporates synergy of multiple teaming strategies to enhance LLMs' resilience. This paper will synthesize the findings from each vulnerability section and propose new directions of research and development. By understanding the focal points of current vulnerabilities, we can better anticipate and mitigate future risks, paving the road for more robust and secure LLMs.

Related papers

• Mitigating Backdoor Threats to Large Language Models: Advancement and Challenges [46.032173498399885]
  Large Language Models (LLMs) have significantly impacted various domains, including Web search, healthcare, and software development. As these models scale, they become more vulnerable to cybersecurity risks, particularly backdoor attacks.
  arXiv  Detail & Related papers  (2024-09-30T06:31:36Z)
• Detecting and Understanding Vulnerabilities in Language Models via Mechanistic Interpretability [44.99833362998488]
  Large Language Models (LLMs) have shown impressive performance across a wide range of tasks. LLMs in particular are known to be vulnerable to adversarial attacks, where an imperceptible change to the input can mislead the output of the model. We propose a method, based on Mechanistic Interpretability (MI) techniques, to guide this process.
  arXiv  Detail & Related papers  (2024-07-29T09:55:34Z)
• Exploring Automatic Cryptographic API Misuse Detection in the Era of LLMs [60.32717556756674]
  This paper introduces a systematic evaluation framework to assess Large Language Models in detecting cryptographic misuses. Our in-depth analysis of 11,940 LLM-generated reports highlights that the inherent instabilities in LLMs can lead to over half of the reports being false positives. The optimized approach achieves a remarkable detection rate of nearly 90%, surpassing traditional methods and uncovering previously unknown misuses in established benchmarks.
  arXiv  Detail & Related papers  (2024-07-23T15:31:26Z)
• A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends [78.3201480023907]
  Large Vision-Language Models (LVLMs) have demonstrated remarkable capabilities across a wide range of multimodal understanding and reasoning tasks. The vulnerability of LVLMs is relatively underexplored, posing potential security risks in daily usage. In this paper, we provide a comprehensive review of the various forms of existing LVLM attacks.
  arXiv  Detail & Related papers  (2024-07-10T06:57:58Z)
• Towards Effectively Detecting and Explaining Vulnerabilities Using Large Language Models [17.96542494363619]
  Large language models (LLMs) have demonstrated remarkable capabilities in comprehending complex contexts. In this paper, we conduct a study to investigate the capabilities of LLMs in both detecting and explaining vulnerabilities. Under specialized fine-tuning for vulnerability explanation, our LLMVulExp not only detects the types of vulnerabilities in the code but also analyzes the code context to generate the cause, location, and repair suggestions.
  arXiv  Detail & Related papers  (2024-06-14T04:01:25Z)
• Understanding Privacy Risks of Embeddings Induced by Large Language Models [75.96257812857554]
  Large language models show early signs of artificial general intelligence but struggle with hallucinations. One promising solution is to store external knowledge as embeddings, aiding LLMs in retrieval-augmented generation. Recent studies experimentally showed that the original text can be partially reconstructed from text embeddings by pre-trained language models.
  arXiv  Detail & Related papers  (2024-04-25T13:10:48Z)
• Unveiling the Misuse Potential of Base Large Language Models via In-Context Learning [61.2224355547598]
  Open-sourcing of large language models (LLMs) accelerates application development, innovation, and scientific progress. Our investigation exposes a critical oversight in this belief. By deploying carefully designed demonstrations, our research demonstrates that base LLMs could effectively interpret and execute malicious instructions.
  arXiv  Detail & Related papers  (2024-04-16T13:22:54Z)
• Fine-Tuning, Quantization, and LLMs: Navigating Unintended Outcomes [0.0]
  Large Language Models (LLMs) have gained widespread adoption across various domains, including chatbots and auto-task completion agents. These models are susceptible to safety vulnerabilities such as jailbreaking, prompt injection, and privacy leakage attacks. This study investigates the impact of these modifications on LLM safety, a critical consideration for building reliable and secure AI systems.
  arXiv  Detail & Related papers  (2024-04-05T20:31:45Z)
• How Far Have We Gone in Vulnerability Detection Using Large Language Models [15.09461331135668]
  We introduce a comprehensive vulnerability benchmark VulBench. This benchmark aggregates high-quality data from a wide range of CTF challenges and real-world applications. We find that several LLMs outperform traditional deep learning approaches in vulnerability detection.
  arXiv  Detail & Related papers  (2023-11-21T08:20:39Z)
• Privacy in Large Language Models: Attacks, Defenses and Future Directions [84.73301039987128]
  We analyze the current privacy attacks targeting large language models (LLMs) and categorize them according to the adversary's assumed capabilities. We present a detailed overview of prominent defense strategies that have been developed to counter these privacy attacks.
  arXiv  Detail & Related papers  (2023-10-16T13:23:54Z)
• On the Risk of Misinformation Pollution with Large Language Models [127.1107824751703]
  We investigate the potential misuse of modern Large Language Models (LLMs) for generating credible-sounding misinformation. Our study reveals that LLMs can act as effective misinformation generators, leading to a significant degradation in the performance of Open-Domain Question Answering (ODQA) systems.
  arXiv  Detail & Related papers  (2023-05-23T04:10:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.