Extrapolating Volition with Recursive Information Markets

• URL: http://arxiv.org/abs/2604.08606v1
• Date: Wed, 08 Apr 2026 13:57:38 GMT
• Title: Extrapolating Volition with Recursive Information Markets
• Abstract summary: We argue that using Large Language Model (LLM) buyers to inspect and purchase information could overcome this information asymmetry.<n>We analyze this mechanism formally through a "value-of-information" paradigm, i.e. whether it incentivizes information to be priced and provided in accordance with its "true value"
• Score: 9.297427304313336
• License: http://creativecommons.org/licenses/by-sa/4.0/
• Abstract: One of the impediments to the efficiency of information markets is the inherent information asymmetry present in them, exacerbated by the "buyer's inspection paradox" (the buyer cannot mitigate the asymmetry by "inspecting" the information, because in doing so the buyer obtains the information without paying for it). Previous work has suggested that using Large Language Model (LLM) buyers to inspect and purchase information could overcome this information asymmetry, as an LLM buyer can simply "forget" the information it inspects. In this work, we analyze this mechanism formally through a "value-of-information" paradigm, i.e. whether it incentivizes information to be priced and provided in accordance with its "true value". We focus in particular on our new recursive version of the mechanism, which we believe has a range of applications including in AI alignment research, where it is related to Extrapolated Volition and Scalable Oversight.

Related papers

• Paying to Know: Micro-Transaction Markets for Verified Product Information in Agentic E-Commerce [4.655159257282136]
  We argue that the arrival of agent-native micro-payment rails changes what is scarce.<n>We envision agentic e-commerce as a micro-transaction market for verified information.
  arXiv  Detail & Related papers  (2026-06-23T16:42:21Z)
• PrivacyPeek: Auditing What LLM-Based Agents Acquire, Not Just What They Say [48.84133320567554]
  LLM-based agents are rapidly advancing, autonomously invoking external tools to complete tasks for users.<n> Existing privacy benchmarks audit what the agent's response or outgoing actions disclose, but overlook the acquisition stage where data first enters the agent's context.<n>We introduce emphPrivacyPeek, a benchmark for evaluating acquisition-stage privacy leakage of LLM-based agents.
  arXiv  Detail & Related papers  (2026-05-29T04:55:12Z)
• Information Aggregation with AI Agents [0.0]
  We conduct a controlled experiment where AI agents trade in a prediction market after receiving private signals.<n>We find that although the median market is effective at aggregating information in the easy information structures, increasing the complexity has a significant and negative impact.<n>We establish that "smarter" AI agents perform better at aggregation and they are more profitable.
  arXiv  Detail & Related papers  (2026-04-21T23:17:24Z)
• When Life Gives You AI, Will You Turn It Into A Market for Lemons? Understanding How Information Asymmetries About AI System Capabilities Affect Market Outcomes and Adoption [45.10829096284761]
  Complex AI systems can appear highly accurate while making costly errors or embedding hidden defects.<n>This paper provides the first experimental evidence on the role of information asymmetries and disclosure designs in shaping user adoption of AI systems.
  arXiv  Detail & Related papers  (2026-01-29T12:49:28Z)
• Balancing Truthfulness and Informativeness with Uncertainty-Aware Instruction Fine-Tuning [79.48839334040197]
  Instruction fine-tuning (IFT) can increase the informativeness of large language models (LLMs), but may reduce their truthfulness.<n>In this paper, we empirically demonstrate how unfamiliar knowledge in IFT datasets can negatively affect the truthfulness of LLMs.<n>We introduce two new IFT paradigms, $UNIT_cut$ and $UNIT_ref$, to address this issue.
  arXiv  Detail & Related papers  (2025-02-17T16:10:30Z)
• Verification of Machine Unlearning is Fragile [48.71651033308842]
  We introduce two novel adversarial unlearning processes capable of circumventing both types of verification strategies. This study highlights the vulnerabilities and limitations in machine unlearning verification, paving the way for further research into the safety of machine unlearning.
  arXiv  Detail & Related papers  (2024-08-01T21:37:10Z)
• On Bits and Bandits: Quantifying the Regret-Information Trade-off [62.64904903955711]
  We study the trade-off between the information an agent accumulates and the regret it suffers.<n>We introduce the first Bayesian regret lower bounds that depend on the information an agent accumulates.<n>We also prove regret upper bounds using the amount of information the agent accumulates.
  arXiv  Detail & Related papers  (2024-05-26T14:18:38Z)
• Language Models Can Reduce Asymmetry in Information Markets [100.38786498942702]
  We introduce an open-source simulated digital marketplace where intelligent agents, powered by language models, buy and sell information on behalf of external participants. The central mechanism enabling this marketplace is the agents' dual capabilities: they have the capacity to assess the quality of privileged information but also come equipped with the ability to forget. To perform well, agents must make rational decisions, strategically explore the marketplace through generated sub-queries, and synthesize answers from purchased information.
  arXiv  Detail & Related papers  (2024-03-21T14:48:37Z)
• Exploitation Business: Leveraging Information Asymmetry [0.0]
  "Exploitation Business" focuses on businesses targeting non-experts or fraudsters who capitalize on information asymmetry to sell their products or services to desperate individuals. This phenomenon thrives on individuals' limited access to information, lack of expertise, and Fear of Missing Out (FOMO) The recent advancement of social media and the rising trend of fandom business have accelerated the proliferation of such exploitation business models.
  arXiv  Detail & Related papers  (2023-10-15T11:18:54Z)
• Bloated Disclosures: Can ChatGPT Help Investors Process Information? [1.3594158136398242]
  Generative AI tools such as ChatGPT can fundamentally change the way investors process information.<n>We probe the economic usefulness of these tools in summarizing complex corporate disclosures using the stock market as a laboratory.<n>We show that bloated disclosure is associated with adverse capital market consequences, such as lower price efficiency and higher information asymmetry.
  arXiv  Detail & Related papers  (2023-06-17T01:22:08Z)
• Information Association for Language Model Updating by Mitigating LM-Logical Discrepancy [68.31760483418901]
  Large Language Models(LLMs) struggle with providing current information due to the outdated pre-training data. Existing methods for updating LLMs, such as knowledge editing and continual fine-tuning, have significant drawbacks in generalizability of new information. We identify the core challenge behind these drawbacks: the LM-logical discrepancy featuring the difference between language modeling probabilities and logical probabilities.
  arXiv  Detail & Related papers  (2023-05-29T19:48:37Z)
• Augmented cross-selling through explainable AI -- a case from energy retailing [0.0]
  We analyze data on 220,185 customers of an energy retailer, predict cross-purchases with up to 86% correctness (AUC), and show that the XAI method SHAP provides explanations that hold for actual buyers. We further outline implications for research in information systems, XAI, and relationship marketing.
  arXiv  Detail & Related papers  (2022-08-24T09:51:52Z)
• A Bayesian Framework for Information-Theoretic Probing [51.98576673620385]
  We argue that probing should be seen as approximating a mutual information. This led to the rather unintuitive conclusion that representations encode exactly the same information about a target task as the original sentences. This paper proposes a new framework to measure what we term Bayesian mutual information.
  arXiv  Detail & Related papers  (2021-09-08T18:08:36Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.