Related papers: Algorithmic pricing with independent learners and relative experience replay

Related papers

Beyond Isolation: Unlocking Reinforcement Learning Component Synergy for Sample-Efficient Continuous Control [59.15149762610843]
Reinforcement learning systems are significantly more complex than other machine learning paradigms.<n>Despite advances in individual algorithmic components, their functional interdependencies remain underexplored.<n>We propose ROSER, an RL framework that coordinates three critical dimensions: Model-based Representation, Optimization Stability, and Experience Replay.
arXiv Detail & Related papers (2026-08-07T10:38:31Z)
LLM-as-a-Coach: Experiential Learning for Non-Verifiable Tasks [89.13372484640392]
We propose Experiential Learning (EL), which repurposes the feedback model from an LLM-as-a-Judge into an LLM-as-a-Coach.<n>EL consistently outperforms rubric-based RL on held-out and unseen open-ended tasks.<n>These findings establish experiential knowledge as a richer and more generalizable learning signal for post-training on non-verifiable tasks.
arXiv Detail & Related papers (2026-07-20T16:08:49Z)
TRACE: A Unified Rollout Budget Allocation Framework for Efficient Agentic Reinforcement Learning [52.67756371231985]
Reinforcement learning with verifiable rewards (RLVR) is a promising approach for enhancing reasoning and agentic behavior in large language models.<n>We introduce Tree Rollout Allocation for Contrastive Exploration (TRACE), a unified rollout allocation framework.<n>Technically, TRACE allocates rollout budget to both prompt roots and intermediate prefixes that are most likely to yield mixed terminal rewards.
arXiv Detail & Related papers (2026-06-09T17:16:03Z)
DNQ: Deep Nash Q-Network for Partially Observable n-Player Games [4.004249151279306]
We propose DNQ, a solver-in-the-loop equilibrium supervision framework for training bidding agents.<n>DNQ alternates between trajectory collection, critic-based payoff estimation, equilibrium, and policy imitation.<n>We focus on a scalable pairwise formulation that greatly reduces equilibrium-solving cost and training time compared with the exact formulation.
arXiv Detail & Related papers (2026-06-04T17:58:01Z)
Memory-Induced Supra-Competitive Outcomes Between Deep Reinforcement Learning Agents in Optimal Trade Execution [0.0]
We study a two-agent Almgren-Chriss liquidation game and examine how learned behavior depends on intra-episode environment feedback.<n>We find that, when agents are given access to intra-episode history, especially recent prices and own past actions, supra-competitive outcomes become substantially more frequent and more persistent.
arXiv Detail & Related papers (2026-05-19T18:03:48Z)
ArenaRL: Scaling RL for Open-Ended Agents via Tournament-based Relative Ranking [84.07076200941474]
ArenaRL is a reinforcement learning paradigm that shifts from pointwise scalar scoring to intra-group relative ranking.<n>We construct an intra-group adversarial arena and devise a tournament-based ranking scheme to obtain stable advantage signals.<n>Experiments show that ArenaRL substantially outperforms standard RL baselines.
arXiv Detail & Related papers (2026-01-10T08:43:07Z)
LLMs Can't Handle Peer Pressure: Crumbling under Multi-Agent Social Interactions [35.71511502901056]
Large language models (LLMs) are increasingly deployed in multi-agent systems as components of collaborative intelligence.<n>We examine how LLMs form trust from previous impressions, resist misinformation, and integrate peer input during interaction.<n>We present KAIROS, a benchmark simulating quiz contests with peer agents of varying reliability.
arXiv Detail & Related papers (2025-08-24T09:58:10Z)
Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoning [87.7836502955847]
We propose a novel self-rewarding reinforcement learning framework to enhance Large Language Model (LLM) reasoning.<n>Our key insight is that correct responses often exhibit consistent trajectory patterns in terms of model likelihood.<n>We introduce CoVo, an intrinsic reward mechanism that integrates Consistency and Volatility via a robust vector-space aggregation strategy.
arXiv Detail & Related papers (2025-06-10T12:40:39Z)
Learning to Charge More: A Theoretical Study of Collusion by Q-Learning Agents [9.053163124987535]
We provide the first theoretical explanation for this behavior in infinite repeated games.<n>We show that when the game admits both a one-stage Nash equilibrium price and a collusive-enabling price, firms learn to consistently charge supracompetitive prices.
arXiv Detail & Related papers (2025-05-28T22:18:35Z)
Learning Equilibria from Data: Provably Efficient Multi-Agent Imitation Learning [69.45910671974296]
We show that a new quantity named the single policy deviation concentrability coefficient is unavoidable in the non-interactive imitation learning setting.<n>We introduce two novel solution algorithms: MAIL-BRO and MURMAIL.<n>The latter bypasses completely the best response oracle at the cost of a worse expert query complexity of order $mathcalO(varepsilon-8)$.
arXiv Detail & Related papers (2025-05-23T08:18:35Z)
Naive Algorithmic Collusion: When Do Bandit Learners Cooperate and When Do They Compete? [0.0]
Algorithmic agents are used in a variety of competitive decision settings. We study the behavior of multi-armed bandit machine learning algorithms used in situations where agents are competing. We show that these context-free bandits, with no knowledge of opponents' choices or outcomes, still will consistently learn collusive behavior.
arXiv Detail & Related papers (2024-11-25T16:58:07Z)
Artificial Intelligence and Algorithmic Price Collusion in Two-sided Markets [9.053163124987535]
We examine how AI agents using Q-learning engage in tacit collusion in two-sided markets. Our experiments reveal that AI-driven platforms achieve higher collusion levels compared to Bertrand competition. Increased network externalities significantly enhance collusion, suggesting AI algorithms exploit them to maximize profits.
arXiv Detail & Related papers (2024-07-04T17:57:56Z)
By Fair Means or Foul: Quantifying Collusion in a Market Simulation with Deep Reinforcement Learning [1.5249435285717095]
This research employs an experimental oligopoly model of repeated price competition. We investigate the strategies and emerging pricing patterns developed by the agents, which may lead to a collusive outcome. Our findings indicate that RL-based AI agents converge to a collusive state characterized by the charging of supracompetitive prices.
arXiv Detail & Related papers (2024-06-04T15:35:08Z)
Nearly Optimal Algorithms for Contextual Dueling Bandits from Adversarial Feedback [58.66941279460248]
Learning from human feedback plays an important role in aligning generative models, such as large language models (LLM) We study a model within this problem domain--contextual dueling bandits with adversarial feedback, where the true preference label can be flipped by an adversary. We propose an algorithm namely robust contextual dueling bandit (algo), which is based on uncertainty-weighted maximum likelihood estimation.
arXiv Detail & Related papers (2024-04-16T17:59:55Z)
Algorithmic Collusion by Large Language Models [0.08192907805418582]
We conduct experiments with algorithmic pricing agents based on Large Language Models (LLMs) We find that LLM-based agents are adept at pricing tasks, autonomously collude in oligopoly settings to the detriment of consumers, and variation in seemingly innocuous phrases in LLM instructions may increase collusion.
arXiv Detail & Related papers (2024-03-31T21:43:05Z)
Compressed Federated Reinforcement Learning with a Generative Model [11.074080383657453]
Reinforcement learning has recently gained unprecedented popularity, yet it still grapples with sample inefficiency. Addressing this challenge, federated reinforcement learning (FedRL) has emerged, wherein agents collaboratively learn a single policy by aggregating local estimations. We propose CompFedRL, a communication-efficient FedRL approach incorporating both textitperiodic aggregation and (direct/error-feedback) compression mechanisms.
arXiv Detail & Related papers (2024-03-26T15:36:47Z)
Tacit algorithmic collusion in deep reinforcement learning guided price competition: A study using EV charge pricing game [0.0]
Players in pricing games with complex structures are increasingly adopting artificial intelligence (AI) aided learning algorithms. Recent studies of games in canonical forms have shown contrasting claims ranging from none to a high level of tacit collusion. We consider a practical game where EV charging hubs compete by dynamically varying their prices. Results from our numerical case study yield collusion index values between 0.14 and 0.45, suggesting a low to moderate level of collusion.
arXiv Detail & Related papers (2024-01-25T16:51:52Z)
Augmenting Unsupervised Reinforcement Learning with Self-Reference [63.68018737038331]
Humans possess the ability to draw on past experiences explicitly when learning new tasks. We propose the Self-Reference (SR) approach, an add-on module explicitly designed to leverage historical information. Our approach achieves state-of-the-art results in terms of Interquartile Mean (IQM) performance and Optimality Gap reduction on the Unsupervised Reinforcement Learning Benchmark.
arXiv Detail & Related papers (2023-11-16T09:07:34Z)
Variance-Aware Regret Bounds for Stochastic Contextual Dueling Bandits [53.281230333364505]
This paper studies the problem of contextual dueling bandits, where the binary comparison of dueling arms is generated from a generalized linear model (GLM) We propose a new SupLinUCB-type algorithm that enjoys computational efficiency and a variance-aware regret bound $tilde Obig(dsqrtsum_t=1Tsigma_t2 + dbig)$. Our regret bound naturally aligns with the intuitive expectation in scenarios where the comparison is deterministic, the algorithm only suffers from an $tilde O(d)$ regret.
arXiv Detail & Related papers (2023-10-02T08:15:52Z)
CLARE: Conservative Model-Based Reward Learning for Offline Inverse Reinforcement Learning [26.05184273238923]
This work aims to tackle a major challenge in offline Inverse Reinforcement Learning (IRL) We devise a principled algorithm (namely CLARE) that solves offline IRL efficiently via integrating "conservatism" into a learned reward function. Our theoretical analysis provides an upper bound on the return gap between the learned policy and the expert policy.
arXiv Detail & Related papers (2023-02-09T17:16:29Z)
Credit-cognisant reinforcement learning for multi-agent cooperation [0.0]
We introduce the concept of credit-cognisant rewards, which allows an agent to perceive the effect its actions had on the environment as well as on its co-agents. We show that by manipulating these experiences and constructing the reward contained within them to include the rewards received by all the agents within the same action sequence, we are able to improve significantly on the performance of independent deep Q-learning.
arXiv Detail & Related papers (2022-11-18T09:00:25Z)
Provably Efficient Fictitious Play Policy Optimization for Zero-Sum Markov Games with Structured Transitions [145.54544979467872]
We propose and analyze new fictitious play policy optimization algorithms for zero-sum Markov games with structured but unknown transitions. We prove tight $widetildemathcalO(sqrtK)$ regret bounds after $K$ episodes in a two-agent competitive game scenario. Our algorithms feature a combination of Upper Confidence Bound (UCB)-type optimism and fictitious play under the scope of simultaneous policy optimization.
arXiv Detail & Related papers (2022-07-25T18:29:16Z)
Simultaneous Double Q-learning with Conservative Advantage Learning for Actor-Critic Methods [133.85604983925282]
We propose Simultaneous Double Q-learning with Conservative Advantage Learning (SDQ-CAL) Our algorithm realizes less biased value estimation and achieves state-of-the-art performance in a range of continuous control benchmark tasks.
arXiv Detail & Related papers (2022-05-08T09:17:16Z)
Convergence Results For Q-Learning With Experience Replay [51.11953997546418]
We provide a convergence rate guarantee, and discuss how it compares to the convergence of Q-learning depending on important parameters such as the frequency and number of iterations of replay. We also provide theoretical evidence showing when we might expect this to strictly improve performance, by introducing and analyzing a simple class of MDPs.
arXiv Detail & Related papers (2021-12-08T10:22:49Z)
Online Search With Best-Price and Query-Based Predictions [2.3204178451683264]
We study learning-augmented algorithms in which there is a potentially erroneous prediction concerning the input. We provide experimental results on data obtained from stock exchange markets.
arXiv Detail & Related papers (2021-12-02T20:18:37Z)
Linear Contextual Bandits with Adversarial Corruptions [91.38793800392108]
We study the linear contextual bandit problem in the presence of adversarial corruption. We present a variance-aware algorithm that is adaptive to the level of adversarial contamination $C$.
arXiv Detail & Related papers (2021-10-25T02:53:24Z)
Towards an Understanding of Benign Overfitting in Neural Networks [104.2956323934544]
Modern machine learning models often employ a huge number of parameters and are typically optimized to have zero training loss. We examine how these benign overfitting phenomena occur in a two-layer neural network setting. We show that it is possible for the two-layer ReLU network interpolator to achieve a near minimax-optimal learning rate.
arXiv Detail & Related papers (2021-06-06T19:08:53Z)
Revisiting Smoothed Online Learning [70.09792747315323]
We investigate the problem of smoothed online learning, in which the online learner suffers both a hitting cost and a switching cost. To bound the competitive ratio, we assume the hitting cost is known to the learner in each round, and investigate the greedy algorithm which simply minimizes the weighted sum of the hitting cost and the switching cost.
arXiv Detail & Related papers (2021-02-13T14:15:55Z)
Accelerated Convergence for Counterfactual Learning to Rank [65.63997193915257]
We show that convergence rate of SGD approaches with IPS-weighted gradients suffers from the large variance introduced by the IPS weights. We propose a novel learning algorithm, called CounterSample, that has provably better convergence than standard IPS-weighted gradient descent methods. We prove that CounterSample converges faster and complement our theoretical findings with empirical results.
arXiv Detail & Related papers (2020-05-21T12:53:36Z)
Greedy Algorithm almost Dominates in Smoothed Contextual Bandits [100.09904315064372]
Online learning algorithms must balance exploration and exploitation. We show that a greedy approach almost matches the best possible Bayesian regret rate of any other algorithm.
arXiv Detail & Related papers (2020-05-19T18:11:40Z)
Multi-Issue Bargaining With Deep Reinforcement Learning [0.0]
This paper evaluates the use of deep reinforcement learning in bargaining games. Two actor-critic networks were trained for the bidding and acceptance strategy. Neural agents learn to exploit time-based agents, achieving clear transitions in decision preference values. They also demonstrate adaptive behavior against different combinations of concession, discount factors, and behavior-based strategies.
arXiv Detail & Related papers (2020-02-18T18:33:46Z)
Adversarial Attacks on Linear Contextual Bandits [87.08004581867537]
Malicious agents may have incentives to attack the bandit algorithm to induce it to perform a desired behavior. We show that a malicious agent can force a linear contextual bandit algorithm to pull any desired arm $T - o(T)$ times over a horizon of $T$ steps. We also investigate the case when a malicious agent is interested in affecting the behavior of the bandit algorithm in a single context.
arXiv Detail & Related papers (2020-02-10T15:04:09Z)

This list is automatically generated from the titles and abstracts of the papers in this site.