Survival of the Safest: Towards Secure Prompt Optimization through Interleaved Multi-Objective Evolution

• URL: http://arxiv.org/abs/2410.09652v1
• Date: Sat, 12 Oct 2024 21:16:29 GMT
• Title: Survival of the Safest: Towards Secure Prompt Optimization through Interleaved Multi-Objective Evolution
• Authors: Ankita Sinha, Wendi Cui, Kamalika Das, Jiaxin Zhang,
• Abstract summary: "Survival of the Safest" (SoS) is an innovative multi-objective prompt optimization framework. It enhances both performance and security in large language models simultaneously. SoS provides a scalable solution that expedites optimization in complex, high-dimensional discrete search spaces.
• Score: 1.8814321586521556
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Large language models (LLMs) have demonstrated remarkable capabilities; however, the optimization of their prompts has historically prioritized performance metrics at the expense of crucial safety and security considerations. To overcome this shortcoming, we introduce "Survival of the Safest" (SoS), an innovative multi-objective prompt optimization framework that enhances both performance and security in LLMs simultaneously. SoS utilizes an interleaved multi-objective evolution strategy, integrating semantic, feedback, and crossover mutations to effectively traverse the prompt landscape. Differing from the computationally demanding Pareto front methods, SoS provides a scalable solution that expedites optimization in complex, high-dimensional discrete search spaces while keeping computational demands low. Our approach accommodates flexible weighting of objectives and generates a pool of optimized candidates, empowering users to select prompts that optimally meet their specific performance and security needs. Experimental evaluations across diverse benchmark datasets affirm SoS's efficacy in delivering high performance and notably enhancing safety and security compared to single-objective methods. This advancement marks a significant stride towards the deployment of LLM systems that are both high-performing and secure across varied industrial applications

Related papers

• Joint Optimization of Prompt Security and System Performance in Edge-Cloud LLM Systems [15.058369477125893]
  Large language models (LLMs) have significantly facilitated human life, and prompt engineering has improved the efficiency of these models. Recent years have witnessed a rise in prompt engineering-empowered attacks, leading to issues such as privacy leaks, increased latency, and system resource wastage. We jointly consider prompt security, service latency, and system resource optimization in Edge-Cloud LLM (EC-LLM) systems under various prompt attacks.
  arXiv  Detail & Related papers  (2025-01-30T14:33:49Z)
• Optima: Optimizing Effectiveness and Efficiency for LLM-Based Multi-Agent System [75.25394449773052]
  Large Language Model (LLM) based multi-agent systems (MAS) show remarkable potential in collaborative problem-solving. Yet they still face critical challenges: low communication efficiency, poor scalability, and a lack of effective parameter-updating optimization methods. We present Optima, a novel framework that addresses these issues by significantly enhancing both communication efficiency and task effectiveness.
  arXiv  Detail & Related papers  (2024-10-10T17:00:06Z)
• GANPrompt: Enhancing Robustness in LLM-Based Recommendations with GAN-Enhanced Diversity Prompts [15.920623515602038]
  Large Language Models (LLMs) are highly susceptible to the influence of prompt words. This paper proposes GANPrompt, a multi-dimensional LLMs prompt diversity framework based on Generative Adversarial Networks (GANs) The framework enhances the model's adaptability and stability to diverse prompts by integrating GANs generation techniques with the deep semantic understanding capabilities of LLMs.
  arXiv  Detail & Related papers  (2024-08-19T03:13:20Z)
• Large Language Model as a Catalyst: A Paradigm Shift in Base Station Siting Optimization [62.16747639440893]
  Large language models (LLMs) and their associated technologies advance, particularly in the realms of prompt engineering and agent engineering. Our proposed framework incorporates retrieval-augmented generation (RAG) to enhance the system's ability to acquire domain-specific knowledge and generate solutions.
  arXiv  Detail & Related papers  (2024-08-07T08:43:32Z)
• RigorLLM: Resilient Guardrails for Large Language Models against Undesired Content [62.685566387625975]
  Current mitigation strategies, while effective, are not resilient under adversarial attacks. This paper introduces Resilient Guardrails for Large Language Models (RigorLLM), a novel framework designed to efficiently moderate harmful and unsafe inputs.
  arXiv  Detail & Related papers  (2024-03-19T07:25:02Z)
• SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models [107.82336341926134]
  SALAD-Bench is a safety benchmark specifically designed for evaluating Large Language Models (LLMs) It transcends conventional benchmarks through its large scale, rich diversity, intricate taxonomy spanning three levels, and versatile functionalities.
  arXiv  Detail & Related papers  (2024-02-07T17:33:54Z)
• Towards Safe Multi-Task Bayesian Optimization [1.3654846342364308]
  Reduced physical models of the system can be incorporated into the optimization process, accelerating it. These models are able to offer an approximation of the actual system, and evaluating them is significantly cheaper. Safety is a crucial criterion for online optimization methods such as Bayesian optimization.
  arXiv  Detail & Related papers  (2023-12-12T13:59:26Z)
• Query-Dependent Prompt Evaluation and Optimization with Offline Inverse RL [62.824464372594576]
  We aim to enhance arithmetic reasoning ability of Large Language Models (LLMs) through zero-shot prompt optimization. We identify a previously overlooked objective of query dependency in such optimization. We introduce Prompt-OIRL, which harnesses offline inverse reinforcement learning to draw insights from offline prompting demonstration data.
  arXiv  Detail & Related papers  (2023-09-13T01:12:52Z)
• A Robust Multi-Objective Bayesian Optimization Framework Considering Input Uncertainty [0.0]
  In real-life applications like engineering design, the designer often wants to take multiple objectives as well as input uncertainty into account. We introduce a novel Bayesian optimization framework to efficiently perform multi-objective optimization considering input uncertainty.
  arXiv  Detail & Related papers  (2022-02-25T17:45:26Z)
• Constrained Policy Optimization via Bayesian World Models [79.0077602277004]
  LAMBDA is a model-based approach for policy optimization in safety critical tasks modeled via constrained Markov decision processes. We demonstrate LAMBDA's state of the art performance on the Safety-Gym benchmark suite in terms of sample efficiency and constraint violation.
  arXiv  Detail & Related papers  (2022-01-24T17:02:22Z)
• Leveraging Trust for Joint Multi-Objective and Multi-Fidelity Optimization [0.0]
  This paper investigates a novel approach to Bayesian multi-objective and multi-fidelity (MOMF) optimization. We suggest the innovative use of a trust metric to support simultaneous optimization of multiple objectives and data sources. Our methods offer broad applicability in solving simulation problems in fields such as plasma physics and fluid dynamics.
  arXiv  Detail & Related papers  (2021-12-27T20:55:26Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.