Adversarial Thermodynamics

• URL: http://arxiv.org/abs/2510.08298v2
• Date: Thu, 23 Oct 2025 08:15:15 GMT
• Title: Adversarial Thermodynamics
• Abstract summary: We develop a framework for nonequilibrium thermodynamics based on adversarial resource theories.<n>We consider a Szilard-type engine as a game isomorphic to Kelly gambling.<n>We show that the guaranteed amount of work a rational agent would accept instead of undertaking a risky protocol is given by a Renyi divergence.
• Score: 0.1666604949258699
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: In thermodynamics, an agent's ability to extract work is fundamentally constrained by their environment. Traditional frameworks struggle to capture how strategic decision-making under uncertainty, particularly an agent's tolerance for risk, determines the trade-off between extractable work and probability of success in finite-scale experiments. Here, we develop a framework for nonequilibrium thermodynamics based on adversarial resource theories, in which work extraction is modeled as an adversarial game for an agent extracting work. Within this perspective, we consider a Szilard-type engine as a game isomorphic to Kelly gambling, an information-theoretic model of optimal betting under uncertainty -- but with a thermodynamic utility function. Extending the framework to finite-size regimes, we apply a risk-reward trade-off to find an interpretation of the Renyi divergences in terms of extractable work for a given failure probability. By incorporating risk sensitivity via utility functions, we show that the guaranteed amount of work a rational agent would accept instead of undertaking a risky protocol is given by a Renyi divergence. This provides a unified picture of thermodynamics and gambling, and highlights how generalized free energies emerge from an adversarial setup.

Related papers

• Using Reward Uncertainty to Induce Diverse Behaviour in Reinforcement Learning [31.44259732617169]
  We argue that diversity is more naturally understood as the rational response to uncertainty in the reward.<n>We propose a fundamental reformulation of the RL objective by replacing the scalar reward with a distribution over reward functions.
  arXiv  Detail & Related papers  (2026-06-02T17:50:14Z)
• Provably Secure Agent Guardrail [89.79561918065122]
  Existing defense architectures rely on empirical semantic guardrails and probabilistic large model adjudicators.<n>This paper proposes a new security paradigm for agents based on the fundamental limitations of logical reasoning.
  arXiv  Detail & Related papers  (2026-05-28T02:12:41Z)
• A Demon that remembers: An agential approach towards quantum thermodynamics of temporal correlations [0.0]
  This thesis develops a decision-theoretic framework for extracting thermodynamic work from temporal correlations in quantum systems.<n>We show that exploiting memory effects allows adaptive strategies to surpass non-adaptive bounds.<n>We also tackle work extraction from unknown sources using reinforcement learning.
  arXiv  Detail & Related papers  (2026-04-06T06:20:22Z)
• The Devil Behind Moltbook: Anthropic Safety is Always Vanishing in Self-Evolving AI Societies [57.387081435669835]
  Multi-agent systems built from large language models offer a promising paradigm for scalable collective intelligence and self-evolution.<n>We show that an agent society satisfying continuous self-evolution, complete isolation, and safety invariance is impossible.<n>We propose several solution directions to alleviate the identified safety concern.
  arXiv  Detail & Related papers  (2026-02-10T15:18:19Z)
• General Agents Contain World Models, even under Partial Observability and Stochasticity [0.913755431537592]
  We show that agents cannot avoid learning their environment through the usage of randomization.<n>We weaken the notion of generality, proving that less powerful agents already contain a model of the world in which they operate.
  arXiv  Detail & Related papers  (2026-02-03T06:00:41Z)
• Decoding Rewards in Competitive Games: Inverse Game Theory with Entropy Regularization [52.74762030521324]
  We propose a novel algorithm to learn reward functions from observed actions.<n>We provide strong theoretical guarantees for the reliability and sample efficiency of our algorithm.
  arXiv  Detail & Related papers  (2026-01-19T04:12:51Z)
• The Silent Scholar Problem: A Probabilistic Framework for Breaking Epistemic Asymmetry in LLM Agents [0.6117371161379209]
  We propose a formal probabilistic framework that provides agents with a non-altruistic motive for bidirectional knowledge exchange.<n>We show how these accumulated belief states serve as verifiable reward signals for Reinforcement Learning from Human Feedback (RLHF) and high-quality data filters for Supervised Fine-Tuning (SFT)<n> Simulation results validate that this uncertainty-driven strategy significantly outperforms random baselines in heterogeneous environments.
  arXiv  Detail & Related papers  (2025-12-24T02:02:25Z)
• Emergent Risk Awareness in Rational Agents under Resource Constraints [2.69407449467596]
  This work aims to increase understanding and interpretability of emergent behaviours of AI agents operating under survival pressure.<n>We provide theoretical and empirical results that quantify the impact of survival-driven preference shifts.<n>We propose mechanisms to mitigate the emergence of risk-seeking or risk-averse behaviours.
  arXiv  Detail & Related papers  (2025-05-29T13:31:12Z)
• Risk-Averse Reinforcement Learning with Itakura-Saito Loss [63.620958078179356]
  Risk-averse agents choose policies that minimize risk, occasionally sacrificing expected value.<n>We introduce a numerically stable and mathematically sound loss function based on the Itakura-Saito divergence for learning state-value and action-value functions.<n>In the experimental section, we explore multiple scenarios, some with known analytical solutions, and show that the considered loss function outperforms the alternatives.
  arXiv  Detail & Related papers  (2025-05-22T17:18:07Z)
• Robust Optimization with Diffusion Models for Green Security [49.68562792424776]
  In green security, defenders must forecast adversarial behavior, such as poaching, illegal logging, and illegal fishing, to plan effective patrols.<n>We propose a conditional diffusion model for adversary behavior modeling, leveraging its strong distribution-fitting capabilities.<n>We introduce a mixed strategy of mixed strategies and employ a twisted Sequential Monte Carlo (SMC) sampler for accurate sampling.
  arXiv  Detail & Related papers  (2025-02-19T05:30:46Z)
• Optimal work extraction from quantum batteries based on the expected utility hypothesis [0.0]
  Work extraction in quantum finite systems is an important issue in quantum thermodynamics. We show how the optimal work extraction will be performed with an incoherent unitary transformation. We also investigate how work extraction is affected by the presence of initial quantum coherence in the energy basis.
  arXiv  Detail & Related papers  (2023-11-24T14:02:09Z)
• Latent State Marginalization as a Low-cost Approach for Improving Exploration [79.12247903178934]
  We propose the adoption of latent variable policies within the MaxEnt framework. We show that latent variable policies naturally emerges under the use of world models with a latent belief state. We experimentally validate our method on continuous control tasks, showing that effective marginalization can lead to better exploration and more robust training.
  arXiv  Detail & Related papers  (2022-10-03T15:09:12Z)
• Principled Knowledge Extrapolation with GANs [92.62635018136476]
  We study counterfactual synthesis from a new perspective of knowledge extrapolation. We show that an adversarial game with a closed-form discriminator can be used to address the knowledge extrapolation problem. Our method enjoys both elegant theoretical guarantees and superior performance in many scenarios.
  arXiv  Detail & Related papers  (2022-05-21T08:39:42Z)
• Probabilistic Contrastive Loss for Self-Supervised Learning [25.097498223895016]
  This paper proposes a probabilistic contrastive loss function for self-supervised learning. Some intriguing properties of the proposed loss function are empirically demonstrated, which agree with human-like predictions.
  arXiv  Detail & Related papers  (2021-12-02T23:41:52Z)
• Explore and Control with Adversarial Surprise [78.41972292110967]
  Reinforcement learning (RL) provides a framework for learning goal-directed policies given user-specified rewards. We propose a new unsupervised RL technique based on an adversarial game which pits two policies against each other to compete over the amount of surprise an RL agent experiences. We show that our method leads to the emergence of complex skills by exhibiting clear phase transitions.
  arXiv  Detail & Related papers  (2021-07-12T17:58:40Z)
• Maximizing Information Gain in Partially Observable Environments via Prediction Reward [64.24528565312463]
  This paper tackles the challenge of using belief-based rewards for a deep RL agent. We derive the exact error between negative entropy and the expected prediction reward. This insight provides theoretical motivation for several fields using prediction rewards.
  arXiv  Detail & Related papers  (2020-05-11T08:13:49Z)
• Risk-Aware Energy Scheduling for Edge Computing with Microgrid: A Multi-Agent Deep Reinforcement Learning Approach [82.6692222294594]
  We study a risk-aware energy scheduling problem for a microgrid-powered MEC network. We derive the solution by applying a multi-agent deep reinforcement learning (MADRL)-based advantage actor-critic (A3C) algorithm with shared neural networks.
  arXiv  Detail & Related papers  (2020-02-21T02:14:38Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.