ContextLens: Modeling Imperfect Privacy and Safety Context for Legal Compliance

• URL: http://arxiv.org/abs/2604.12308v1
• Date: Tue, 14 Apr 2026 05:35:38 GMT
• Title: ContextLens: Modeling Imperfect Privacy and Safety Context for Legal Compliance
• Abstract summary: We propose ContextLens, a semi-rule-based framework that leverages large language models (LLMs) to ground input context in the legal domain.<n>We conduct experiments on existing compliance benchmarks that cover the General Data Protection Regulation () and the EU AI Act.<n>The results suggest that our ContextLens can significantly improve LLMs' compliance assessment and surpass existing baselines without any training.
• Score: 49.524070843587594
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Individuals' concerns about data privacy and AI safety are highly contextualized and extend beyond sensitive patterns. Addressing these issues requires reasoning about the context to identify and mitigate potential risks. Though researchers have widely explored using large language models (LLMs) as evaluators for contextualized safety and privacy assessments, these efforts typically assume the availability of complete and clear context, whereas real-world contexts tend to be ambiguous and incomplete. In this paper, we propose ContextLens, a semi-rule-based framework that leverages LLMs to ground the input context in the legal domain and explicitly identify both known and unknown factors for legal compliance. Instead of directly assessing safety outcomes, our ContextLens instructs LLMs to answer a set of crafted questions that span over applicability, general principles and detailed provisions to assess compliance with pre-defined priorities and rules. We conduct extensive experiments on existing compliance benchmarks that cover the General Data Protection Regulation (GDPR) and the EU AI Act. The results suggest that our ContextLens can significantly improve LLMs' compliance assessment and surpass existing baselines without any training. Additionally, our ContextLens can further identify the ambiguous and missing factors.

Related papers

• Reinforcing privacy reasoning in LLMs via normative simulacra from fiction [1.143869785127334]
  Contextual Integrity provides a principled framework, defining privacy as the appropriate flow of information within context-relative norms.<n>We propose extracting normative simulacra from fiction novels and using them to fine-tune LLMs.<n>We evaluate on five CI-aligned benchmarks spanning distinct societal contexts.
  arXiv  Detail & Related papers  (2026-04-21T19:16:22Z)
• Doc-PP: Document Policy Preservation Benchmark for Large Vision-Language Models [13.70855540464427]
  We introduce Doc-PP, a novel benchmark constructed from real-world reports requiring reasoning across heterogeneous visual and textual elements under strict non-disclosure policies.<n>Our evaluation highlights a systemic Reasoning-Induced Safety Gap: models frequently leak sensitive information when answers must be inferred through complex synthesis or aggregated across modalities.<n>We propose DVA, a structural inference framework that decouples reasoning from policy verification.
  arXiv  Detail & Related papers  (2026-01-07T13:45:39Z)
• ROSE: Toward Reality-Oriented Safety Evaluation of Large Language Models [60.28667314609623]
  Large Language Models (LLMs) are increasingly deployed as black-box components in real-world applications.<n>We propose Reality-Oriented Safety Evaluation (ROSE), a novel framework that uses multi-objective reinforcement learning to fine-tune an adversarial LLM.
  arXiv  Detail & Related papers  (2025-06-17T10:55:17Z)
• Keep Security! Benchmarking Security Policy Preservation in Large Language Model Contexts Against Indirect Attacks in Question Answering [15.506092447902931]
  Large Language Models (LLMs) are increasingly deployed in sensitive domains such as enterprise and government.<n>We introduce a novel large-scale benchmark dataset, CoPriva, evaluating LLM adherence to contextual non-disclosure policies in question answering.<n>We evaluate 10 LLMs on our benchmark and reveal a significant vulnerability: many models violate user-defined policies and leak sensitive information.
  arXiv  Detail & Related papers  (2025-05-21T17:58:11Z)
• Context Reasoner: Incentivizing Reasoning Capability for Contextualized Privacy and Safety Compliance via Reinforcement Learning [53.92712851223158]
  We formulate safety and privacy issues into contextualized compliance problems following the Contextual Integrity (CI) theory.<n>Under the CI framework, we align our model with three critical regulatory standards: EU AI Act, and HIPAA.<n>We employ reinforcement learning (RL) with a rule-based reward to incentivize contextual reasoning capabilities while enhancing compliance with safety and privacy norms.
  arXiv  Detail & Related papers  (2025-05-20T16:40:09Z)
• Disambiguation in Conversational Question Answering in the Era of LLMs and Agents: A Survey [54.90240495777929]
  Ambiguity remains a fundamental challenge in Natural Language Processing (NLP)<n>With the advent of Large Language Models (LLMs), addressing ambiguity has become even more critical due to their expanded capabilities and applications.<n>This paper explores the definition, forms, and implications of ambiguity for language driven systems.
  arXiv  Detail & Related papers  (2025-05-18T20:53:41Z)
• CASE-Bench: Context-Aware SafEty Benchmark for Large Language Models [12.385397523940277]
  We introduce CASE-Bench, a Context-Aware SafEty Benchmark that integrates context into safety assessments of large language models (LLMs)<n>Case-Bench assigns distinct, formally described contexts to categorized queries based on Contextual Integrity theory.<n>Our analysis reveals a substantial and significant influence of context on human judgments, underscoring the necessity of context in safety evaluations.
  arXiv  Detail & Related papers  (2025-01-24T21:55:14Z)
• Unveiling the Misuse Potential of Base Large Language Models via In-Context Learning [61.2224355547598]
  Open-sourcing of large language models (LLMs) accelerates application development, innovation, and scientific progress. Our investigation exposes a critical oversight in this belief. By deploying carefully designed demonstrations, our research demonstrates that base LLMs could effectively interpret and execute malicious instructions.
  arXiv  Detail & Related papers  (2024-04-16T13:22:54Z)
• ALERT: A Comprehensive Benchmark for Assessing Large Language Models' Safety through Red Teaming [64.86326523181553]
  ALERT is a large-scale benchmark to assess safety based on a novel fine-grained risk taxonomy. It aims to identify vulnerabilities, inform improvements, and enhance the overall safety of the language models.
  arXiv  Detail & Related papers  (2024-04-06T15:01:47Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.