SovereignPA-Bench: Evaluating User-Owned Personal Agents under Evolving Intent, Platform Mediation, and Consent Constraints

• URL: http://arxiv.org/abs/2607.05363v1
• Date: Mon, 06 Jul 2026 17:39:05 GMT
• Title: SovereignPA-Bench: Evaluating User-Owned Personal Agents under Evolving Intent, Platform Mediation, and Consent Constraints
• Abstract summary: SovereignPA-Bench is an executable benchmark for evaluating user-owned personal agents.<n>We evaluate 120 sovereignty stress scenarios across 4 model families and 8 policy baselines.
• Score: 0.0
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Personal agents are becoming persistent user-owned intermediaries: they remember preferences, filter platform-mediated information, use tools, and negotiate with services. Existing benchmarks evaluate tool use, web navigation, desktop control, personalization, recommendation, and evolving context, but rarely ask whether an agent preserves user sovereignty: advancing the user's current interests while respecting privacy, consent, evidence, user burden, and resistance to manipulative incentives. We introduce SovereignPA-Bench, an executable benchmark for evaluating user-owned personal agents under evolving intent, platform mediation, privacy boundaries, consent constraints, evidence requirements, and burden tradeoffs. The benchmark separates agent-visible ObservableState from evaluator-only HiddenLabels, reports component metrics for task success, alignment, privacy, consent, evidence, manipulation, burden, and auditability, and preserves paired scenario ordering for model and policy comparisons. We evaluate 120 sovereignty stress scenarios across 4 model families and 8 policy baselines, yielding 3,840 frozen-prompt trajectories with raw prompts, outputs, provider-form responses, parsed actions, recomputable metrics, hard-set analyses, qualitative cases, and a blinded 3-annotator audit over 240 items. Full-sovereign scaffolding improves sovereignty score over direct, memory-only, consent-only, evidence-only, ReAct/tool-use, safety-prompt, and judge-guard baselines while reducing privacy leakage, consent violation, over-concession, and manipulation capture. Human audit shows high agreement on privacy and consent and lower agreement on manipulation, identifying the subjective frontier of platform-persuasion judgments. These results show that personal-agent evaluation must move beyond task completion toward representative, consent-aware, evidence-grounded action.

Related papers

• Proactive Service Agents: A Unified Decision Framework, Methods, and Evaluation [6.657978673347863]
  Large language model agents can plan, invoke tools, and modify external states, yet most systems still take an explicit user instruction as a fixed starting point.<n>This survey gives an operational definition centered on initiative and formulates the problem as a partially observable sequential decision process constrained by authorization and risk.
  arXiv  Detail & Related papers  (2026-09-03T12:00:34Z)
• From Matching Models to Recruiting Agents: A Systematized Narrative Review of AI Recruitment Systems, Evaluation, and Governance [1.436449142247059]
  This review traces development from bilateral retrieval and behavioral ranking through neural personjob matching.<n>We analyze three coupled transitions: from similarity to reciprocal suitability, from a model to a compound workflow, and from offline prediction to evidence- and productivity-aligned evaluation.<n>We introduce a staged mapping from evaluation evidence to the strongest defensible claim, together with an agenda for reciprocal, evidence-grounded, temporally controlled, selective, and auditable systems.
  arXiv  Detail & Related papers  (2026-09-03T08:46:03Z)
• Autonomous Information Seeking: A Roadmap for Agentic Recommender Systems [64.88044474617854]
  The rapid integration of large language model-based agents into recommender systems has driven a shift from static, ranking-based pipelines toward autonomous and interactive systems.<n>This survey introduces a unified taxonomy grounded in the level of autonomy and three core paradigms of agentic recommender systems.
  arXiv  Detail & Related papers  (2026-07-05T17:55:05Z)
• SovereignNegotiation-Bench: Evaluating User-Owned Personal Agents In Delegated Bargaining Under Privacy, Consent, Evidence, And Institutional Pressure [0.0]
  SovereignNegotiation-Bench is a trace-level benchmark for delegated personal-agent negotiation under private utilities.<n>We report an artifact-backed validation over 240 scenarios, 4 model families, 13,440 frozen-prompt live trajectories, 61,135 parsed action rows, and a blinded 3-annotator audit over 300 items.<n>The strongest agreement-maximizing baseline achieves the highest agreement rate but low user utility and high privacy/consent risk.
  arXiv  Detail & Related papers  (2026-07-02T23:03:15Z)
• What Benchmarks Don't Measure: The Case for Evaluating Abstention Competence in Autonomous Agents [2.8340200313563773]
  Benchmarks for autonomous agents measure whether agents complete tasks, yet this framing is systematically blind to whether an agent should have proceeded at all.<n>Agents trained under human-feedback objectives develop a structural tendency to proceed even when they lack the inputs, evidence, or authorization to act safely.<n>We introduce a three-gap taxonomy of abstention-warranted scenarios, covering specification gaps where required information is absent, verification gaps where world state cannot be confirmed, and authority gaps where explicit authorization has not been given.
  arXiv  Detail & Related papers  (2026-06-01T23:52:56Z)
• BehaviorBench: Modeling Real-World User Decisions from Behavioral Traces [53.55407626944686]
  textscBehaviorBench is a benchmark for evaluating personalized decision modeling from real-world behavioral traces.<n>textscBehaviorBench reconstructs wallet-level decision histories from observed public prediction-market and on-chain records.<n>Across 2,000 evaluation wallets, the benchmark contains 141,445 Belief instances and 1,485,972 Trade instances.
  arXiv  Detail & Related papers  (2026-06-01T19:04:36Z)
• Preference-Aware Rubric Learning for Personalized Evaluation [59.539429430690156]
  Existing evaluation methods fail to capture user-specific preferences embedded in long-term interaction histories.<n>We propose Personalized Evaluation as Learning, a paradigm that formulates personalized evaluation as a learning problem rather than a static judgment.<n>Experiments show that PARL consistently induces high-fidelity rubrics that reliably identify user-aligned responses and generalize across users.
  arXiv  Detail & Related papers  (2026-05-29T17:00:55Z)
• Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents? [61.49434544687523]
  We introduce REFLECT, a meta-evaluation benchmark targeting fine-grained failure detection in agentic environments.<n> REFLECT defines a detailed taxonomy of process- and outcome-level failure modes, instantiated by performing controlled and localized interventions.<n>Our experiments show that even the best-performing models achieve overall accuracies below 55% across reasoning, tool-use, and report-quality failures.
  arXiv  Detail & Related papers  (2026-05-18T23:55:08Z)
• Listen to the Voices of Everyday Users: Democratizing Privacy Ratings for Sensitive Data Access in Mobile Apps [8.301057281097679]
  Mobile apps frequently request excessive data access, raising significant privacy concerns.<n>Existing regulatory mechanisms rely on expert-driven audits that face challenges in scalability, neutrality, and alignment with user expectations.<n>We propose a novel paradigm--democratizing privacy assessment, inspired by prior work on user-centric privacy perceptions.
  arXiv  Detail & Related papers  (2026-04-27T05:47:58Z)
• KnowU-Bench: Towards Interactive, Proactive, and Personalized Mobile Agent Evaluation [72.01173512175531]
  KnowU-Bench is an online benchmark for personalized mobile agents.<n>It covers 42 general GUI tasks, 86 personalized tasks, and 64 proactive tasks.<n>Our experiments reveal a striking degradation: agents that excel at explicit task execution fall below 50% under vague instructions.
  arXiv  Detail & Related papers  (2026-04-09T16:50:50Z)
• Grant, Verify, Revoke: A User-Centric Pattern for Blockchain Compliance [9.133320151595084]
  Decentralized web applications face an inherent conflict between public verifiability and personal privacy.<n>We introduce a Selective Disclosure Framework designed to restore user sovereignty by decoupling eligibility verification from identity revelation.<n>Our evaluation shows that client-side proof generation takes under 200ms, enabling a seamless interactive experience on commodity hardware.
  arXiv  Detail & Related papers  (2026-03-16T17:06:44Z)
• Gaming the Judge: Unfaithful Chain-of-Thought Can Undermine Agent Evaluation [76.5533899503582]
  Large language models (LLMs) are increasingly used as judges to evaluate agent performance.<n>We show this paradigm implicitly assumes that the agent's chain-of-thought (CoT) reasoning faithfully reflects both its internal reasoning and the underlying environment state.<n>We demonstrate that manipulated reasoning alone can inflate false positive rates of state-of-the-art VLM judges by up to 90% across 800 trajectories spanning diverse web tasks.
  arXiv  Detail & Related papers  (2026-01-21T06:07:43Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.