Efficient RL-based Cache Vulnerability Exploration by Penalizing Useless Agent Actions

• URL: http://arxiv.org/abs/2506.07200v1
• Date: Sun, 08 Jun 2025 15:48:16 GMT
• Title: Efficient RL-based Cache Vulnerability Exploration by Penalizing Useless Agent Actions
• Abstract summary: Cache-timing attacks exploit microarchitectural characteristics to leak sensitive data.<n>A method based on Reinforcement Learning (RL) has been proposed to automatically explore vulnerabilities for a given cache structure.<n>We propose a method to identify these useless actions during training and penalize them so that the agent avoids them and the exploration efficiency is improved.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Cache-timing attacks exploit microarchitectural characteristics to leak sensitive data, posing a severe threat to modern systems. Despite its severity, analyzing the vulnerability of a given cache structure against cache-timing attacks is challenging. To this end, a method based on Reinforcement Learning (RL) has been proposed to automatically explore vulnerabilities for a given cache structure. However, a naive RL-based approach suffers from inefficiencies due to the agent performing actions that do not contribute to the exploration. In this paper, we propose a method to identify these useless actions during training and penalize them so that the agent avoids them and the exploration efficiency is improved. Experiments on 17 cache structures show that our training mechanism reduces the number of useless actions by up to 43.08%. This resulted in the reduction of training time by 28\% in the base case and 4.84\% in the geomean compared to a naive RL-based approach.

Related papers

• CacheRL:Multi-Turn Tool-Calling Agents via Cached Rollouts and Hybrid Reward [17.939095034578035]
  CacheRL is a system for training small agent foundation models.<n>It achieves 92 percent process accuracy on multi-step tool-calling tasks.<n>It approaches GPT-5's 94 percent while requiring 100 times less compute.
  arXiv  Detail & Related papers  (2026-06-12T07:01:50Z)
• Plan2Cleanse: Test-Time Backdoor Defense via Monte-Carlo Planning in Deep Reinforcement Learning [12.26506262764069]
  Plan2Cleanse is a test-time detection and mitigation framework.<n>It adapts Monte Carlo Tree Search to efficiently identify and neutralize RL backdoor attacks.<n>Plan2Cleanse achieves substantial improvements in trigger detection success rates.
  arXiv  Detail & Related papers  (2026-05-10T16:34:16Z)
• From Similarity to Vulnerability: Key Collision Attack on LLM Semantic Caching [7.164841206695704]
  We present the first systematic study of integrity risks arising from cache collisions.<n>We introduce CacheAttack, an automated framework for launching black-box collision attacks.<n>A case study on a financial agent illustrates the real-world impact of these vulnerabilities.
  arXiv  Detail & Related papers  (2026-01-30T15:37:00Z)
• AI Security Beyond Core Domains: Resume Screening as a Case Study of Adversarial Vulnerabilities in Specialized LLM Applications [71.27518152526686]
  Large Language Models (LLMs) excel at text comprehension and generation, making them ideal for automated tasks like code review and content moderation.<n>LLMs can be manipulated by "adversarial instructions" hidden in input data, such as resumes or code, causing them to deviate from their intended task.<n>This paper introduces a benchmark to assess this vulnerability in resume screening, revealing attack success rates exceeding 80% for certain attack types.
  arXiv  Detail & Related papers  (2025-12-23T08:42:09Z)
• Exposing Vulnerabilities in RL: A Novel Stealthy Backdoor Attack through Reward Poisoning [5.764005502656677]
  We study a stealthy backdoor attack that manipulates an agent's policy by poisoning its reward signals.<n>The effectiveness of this attack highlights a critical threat to the integrity of deployed RL systems.
  arXiv  Detail & Related papers  (2025-11-27T12:48:33Z)
• Mind the Gap: Time-of-Check to Time-of-Use Vulnerabilities in LLM-Enabled Agents [4.303444472156151]
  Large Language Model (LLM)-enabled agents are rapidly emerging across a wide range of applications.<n>This work presents the first study of time-of-check to time-of-use (TOCTOU) vulnerabilities in LLM-enabled agents.<n>We introduce TOCTOU-Bench, a benchmark with 66 realistic user tasks designed to evaluate this class of vulnerabilities.
  arXiv  Detail & Related papers  (2025-08-23T22:41:49Z)
• Fight Fire with Fire: Defending Against Malicious RL Fine-Tuning via Reward Neutralization [0.0]
  Malicious RL fine-tuning dismantles safety guardrails with remarkable efficiency.<n>Existing defenses targeting supervised fine-tuning prove ineffective.<n>We introduce Reward Neutralization, the first defense framework specifically designed against RL fine-tuning attacks.
  arXiv  Detail & Related papers  (2025-05-07T17:18:48Z)
• Efficient Backdoor Defense in Multimodal Contrastive Learning: A Token-Level Unlearning Method for Mitigating Threats [52.94388672185062]
  We propose an efficient defense mechanism against backdoor threats using a concept known as machine unlearning. This entails strategically creating a small set of poisoned samples to aid the model's rapid unlearning of backdoor vulnerabilities. In the backdoor unlearning process, we present a novel token-based portion unlearning training regime.
  arXiv  Detail & Related papers  (2024-09-29T02:55:38Z)
• Unlearning Backdoor Threats: Enhancing Backdoor Defense in Multimodal Contrastive Learning via Local Token Unlearning [49.242828934501986]
  Multimodal contrastive learning has emerged as a powerful paradigm for building high-quality features. backdoor attacks subtly embed malicious behaviors within the model during training. We introduce an innovative token-based localized forgetting training regime.
  arXiv  Detail & Related papers  (2024-03-24T18:33:15Z)
• Adversarial training for tabular data with attack propagation [1.9826772682131455]
  We propose a new form of adversarial training where attacks are propagated between the two spaces in the training loop. We show that our method can prevent about 30% performance drops under moderate attacks and is essential under very aggressive attacks.
  arXiv  Detail & Related papers  (2023-07-28T17:12:46Z)
• Efficient Backdoor Attacks for Deep Neural Networks in Real-world Scenarios [17.928013313779516]
  Recent deep neural networks (DNNs) have come to rely on vast amounts of training data. In this paper, we introduce a more realistic attack scenario where victims collect data from multiple sources. We introduce three CLIP-based technologies from two distinct streams: Clean Feature Suppression and Poisoning Feature Augmentation.
  arXiv  Detail & Related papers  (2023-06-14T09:21:48Z)
• Backdoor Attack with Sparse and Invisible Trigger [57.41876708712008]
  Deep neural networks (DNNs) are vulnerable to backdoor attacks. backdoor attack is an emerging yet threatening training-phase threat. We propose a sparse and invisible backdoor attack (SIBA)
  arXiv  Detail & Related papers  (2023-05-11T10:05:57Z)
• Backdoor Defense via Suppressing Model Shortcuts [91.30995749139012]
  In this paper, we explore the backdoor mechanism from the angle of the model structure. We demonstrate that the attack success rate (ASR) decreases significantly when reducing the outputs of some key skip connections.
  arXiv  Detail & Related papers  (2022-11-02T15:39:19Z)
• Accelerating Deep Learning Classification with Error-controlled Approximate-key Caching [72.50506500576746]
  We propose a novel caching paradigm, that we named approximate-key caching. While approximate cache hits alleviate DL inference workload and increase the system throughput, they however introduce an approximation error. We analytically model our caching system performance for classic LRU and ideal caches, we perform a trace-driven evaluation of the expected performance, and we compare the benefits of our proposed approach with the state-of-the-art similarity caching.
  arXiv  Detail & Related papers  (2021-12-13T13:49:11Z)
• Robust Deep Reinforcement Learning through Adversarial Loss [74.20501663956604]
  Recent studies have shown that deep reinforcement learning agents are vulnerable to small adversarial perturbations on the agent's inputs. We propose RADIAL-RL, a principled framework to train reinforcement learning agents with improved robustness against adversarial attacks.
  arXiv  Detail & Related papers  (2020-08-05T07:49:42Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.