Evaluation of Alternative-Based Information Systems for Deliberative Polling using an Agentic Simulator

• URL: http://arxiv.org/abs/2606.11692v1
• Date: Wed, 10 Jun 2026 06:15:13 GMT
• Title: Evaluation of Alternative-Based Information Systems for Deliberative Polling using an Agentic Simulator
• Abstract summary: Deliberative polling promises to improve collective decision-making by exposing shareholders to a broad range of arguments before they vote.<n>Yet ensuring that every voter encounters a representative sample of the reason space, the coverage problem remains an open challenge.<n>This paper introduces a way of evaluating solutions using the LLM-based Agentic Bipolar Argumentation Simulator.
• Score: 0.7354329172946913
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Deliberative polling promises to improve collective decision-making by exposing shareholders to a broad range of arguments before they vote. Yet ensuring that every voter encounters a representative sample of the reason space, the coverage problem, remains an open challenge, particularly at scale and in adversarial or strategically motivated electorates. This paper introduces a way of evaluating solutions using the LLM-based Agentic Bipolar Argumentation Simulator, grounded in a framework which formalises a poll as a six-tuple <Jend, Jopp, Ratt, Renh, VA, VR> of endorsing and opposing justifications, attack and enhance relations, and shareholder- and relation-weights. ABAS simulates N autonomous shareholder agents, each assigned a latent opinion according to desired distributions in [-1, 1], who sequentially vote, choose or author justifications, and optionally submit argumentation-graph links. The simulator implements recommendations that rank existing justifications by their observable endorsement mass. It evaluates the mechanism's success by coverage, namely the fraction of the corpus reason-tag set represented in the K recommendations presented to each shareholder, as a solution to the NP-hard Subsuming Justification Problem. Reported experiments characterise how creativity rate (pown), recommendation size (K), argumentation density (plinks), and population size (N) affect coverage and corpus diversity. In an authenticated electorate where Sybil attacks are impossible and only the relation graph is gameable, we stress-test the scoring with coordinated strategic voting attacks: a tag-flood attack collapses coverage, while author-count relation weighting through a reversed-PageRank rule resists the flood markedly better than uniform weights.

Related papers

• Co-Evolving LLM Evaluators and Policies via DynamicRubric [67.2962847914162]
  Post-training with evaluator feedback on policy-induced samples serves as a major mechanism for improving large language models.<n>We propose DynamicRubric, a response-set-conditioned evaluator--policy co-evolution framework.<n>In experiments with 8B backbones, DynamicRubric improves evaluator performance and provides stronger policy supervision than baselines.
  arXiv  Detail & Related papers  (2026-07-22T12:35:40Z)
• Beyond Rating: A Comprehensive Evaluation and Benchmark for AI Reviews [69.66583722746904]
  We introduce Beyond Rating, a holistic evaluation framework that assesses AI reviewers across five dimensions.<n>We propose a Max-Recall strategy to accommodate valid expert disagreement and introduce a curated dataset of paper with high-confidence reviews.<n>Our proposed text-centric metrics--particularly the recall of weakness arguments--correlate strongly with rating accuracy.
  arXiv  Detail & Related papers  (2026-04-21T14:21:15Z)
• DARC: Disagreement-Aware Alignment via Risk-Constrained Decoding [59.16244104797919]
  We propose **Disagreement-Aware Alignment via Risk-Constrained Decoding (DARC)**.<n>DARC frames response selection as distributionally robust, risk-sensitive decision making.<n>Experiments on alignment benchmarks show that DARC reduces disagreement and tail risk while maintaining competitive average quality.
  arXiv  Detail & Related papers  (2026-03-09T09:21:29Z)
• A Single Revision Step Improves Token-Efficient LLM Reasoning [3.344806691289323]
  We introduce Packet-Conditioned Revision (PACER), a training-free, inference-only framework for large language models.<n>PACER enables reasoning traces to revise their conclusions through a structured coordination step.<n>On challenging competitive math benchmarks, PACER matches or exceeds the accuracy of 256-sample majority voting.
  arXiv  Detail & Related papers  (2026-02-02T21:28:42Z)
• Selective Adversarial Attacks on LLM Benchmarks [1.6307653659652344]
  We study selective adversarial attacks on the widely used benchmark MMLU.<n>We find that selective adversarial attacks exist and can materially alter relative rankings.<n>Our results motivate perturbation-aware reporting and robustness evaluation.
  arXiv  Detail & Related papers  (2025-10-15T14:08:44Z)
• Reference-Free Rating of LLM Responses via Latent Information [53.463883683503106]
  We study the common practice of asking a judge model to assign Likert-scale scores to free-text responses.<n>We then propose and evaluate Latent Judges, which derive scalar ratings from internal model signals.<n>Across a broad suite of pairwise and single-rating benchmarks, latent methods match or surpass standard prompting.
  arXiv  Detail & Related papers  (2025-09-29T12:15:52Z)
• Beyond RLHF and NLHF: Population-Proportional Alignment under an Axiomatic Framework [7.065259679465175]
  We develop a novel preference learning framework capable of aligning aggregate opinions and policies proportionally with the true population distribution of evaluator preferences.<n>We propose a soft-max relaxation method that smoothly trade-offs population-proportional alignment with the selection of the Condorcet winner.
  arXiv  Detail & Related papers  (2025-06-05T22:15:07Z)
• Sequential Manipulation Against Rank Aggregation: Theory and Algorithm [119.57122943187086]
  We leverage an online attack on the vulnerable data collection process. From the game-theoretic perspective, the confrontation scenario is formulated as a distributionally robust game. The proposed method manipulates the results of rank aggregation methods in a sequential manner.
  arXiv  Detail & Related papers  (2024-07-02T03:31:21Z)
• Aligning Large Language Models by On-Policy Self-Judgment [49.31895979525054]
  Existing approaches for aligning large language models with human preferences face a trade-off that requires a separate reward model (RM) for on-policy learning. We present a novel alignment framework, SELF-JUDGE, that does on-policy learning and is parameter efficient. We show that the rejecting sampling by itself can improve performance further without an additional evaluator.
  arXiv  Detail & Related papers  (2024-02-17T11:25:26Z)
• FedQV: Leveraging Quadratic Voting in Federated Learning [2.4554686192257424]
  We propose FedQV, a novel aggregation algorithm built upon the quadratic voting scheme. We show that FedQV can be easily combined with Byzantine-robust privacy-preserving mechanisms to enhance its robustness against both poisoning and privacy attacks.
  arXiv  Detail & Related papers  (2024-01-02T11:53:06Z)
• Piecewise-Stationary Combinatorial Semi-Bandit with Causally Related Rewards [5.347237827669861]
  We study the piecewise stationary semi-bandit problem with causally related rewards. In our nonstationary environment, variations in the base arms' distributions, causal relationships between rewards, or both, change the reward generation process. The problem becomes aggravated in the semi-bandit setting, where the decision-maker only observes the outcome of the selected bundle of arms.
  arXiv  Detail & Related papers  (2023-07-26T12:06:13Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.