From Novice to Expert: Cost-Aware Bandits for Evolving Worker Performance in Crowdsensing

• URL: http://arxiv.org/abs/2607.13546v1
• Date: Wed, 15 Jul 2026 07:51:14 GMT
• Title: From Novice to Expert: Cost-Aware Bandits for Evolving Worker Performance in Crowdsensing
• Abstract summary: We study a budget-constrained online recruitment problem in which the platform selects one worker in each round.<n>We develop a cost-aware online learning framework that jointly learns evolving reward trajectories and heterogeneous costs.
• Score: 11.284703747922757
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Mobile crowdsensing (MC) recruits mobile users to perform sensing tasks using their smartphones, enabling large-scale applications such as traffic monitoring and environmental sensing. A fundamental challenge is online worker recruitment under uncertainty, where the platform must learn workers' sensing performance while operating with a limited budget. Existing learning-based MC recruitment methods typically assume that each worker's sensing quality is stationary with a fixed mean over time. In practice, however, worker performance often improves with experience and eventually stabilizes, while the incurred sensing cost can be unknown in advance due to time-varying device and context states. In this paper, we study a budget-constrained online recruitment problem in which the platform selects one worker in each round, observes the sensing quality and incurred cost, where the expected sensing quality of each worker increases with experience and eventually converges to a plateau, and repeats until the budget is exhausted. We formulate this problem as a structured bandit model where each worker's expected reward evolves according to an unknown increasing-then-converging function of its participation count, and each worker has an unknown expected cost. We develop a cost-aware online learning framework that jointly learns evolving reward trajectories and heterogeneous costs, detects performance saturation, and allocates the limited budget to maximize long-term sensing utility. We provide theoretical performance guarantees and validate the proposed approach through extensive experiments, demonstrating consistent improvements over baselines that ignore experience-driven dynamics or assume known costs.

Related papers

• Better, Faster, Stronger: Programmatic Skill Learning Best Reduces Agent Cost [31.000338964323248]
  We argue that among all the different skill learning methods, those that view skills as programs can achieve the best cost reduction.<n>SpeedRunner is a coding agent that analyzes trajectories and skills for better performance on future tasks.
  arXiv  Detail & Related papers  (2026-08-11T18:42:23Z)
• External Experience Serving in Production LLM Systems: A Deployment-Oriented Study of Quality-Cost Trade-offs [3.542522302754046]
  Injecting external experience can improve task quality, yet it also increases prompt burden, latency, and serving pressure.<n>We compare no-experience baselines, random experience controls, global prompt injection, and retrieval-based selective injection.
  arXiv  Detail & Related papers  (2026-06-10T08:38:55Z)
• RRO: LLM Agent Optimization Through Rising Reward Trajectories [52.579992804584464]
  Large language models (LLMs) have exhibited extraordinary performance in a variety of tasks.<n>In practice, agents sensitive to the outcome of certain key steps which makes them likely to fail the task.<n>We propose Reward Rising Optimization (RRO) to mitigate this issue.
  arXiv  Detail & Related papers  (2025-05-27T05:27:54Z)
• TransPath: Learning Heuristics For Grid-Based Pathfinding via Transformers [64.88759709443819]
  We suggest learning the instance-dependent proxies that are supposed to notably increase the efficiency of the search. The first proxy we suggest to learn is the correction factor, i.e. the ratio between the instance independent cost-to-go estimate and the perfect one. The second proxy is the path probability, which indicates how likely the grid cell is lying on the shortest path.
  arXiv  Detail & Related papers  (2022-12-22T14:26:11Z)
• Actively Learning Costly Reward Functions for Reinforcement Learning [56.34005280792013]
  We show that it is possible to train agents in complex real-world environments orders of magnitudes faster. By enabling the application of reinforcement learning methods to new domains, we show that we can find interesting and non-trivial solutions.
  arXiv  Detail & Related papers  (2022-11-23T19:17:20Z)
• Skill-Based Reinforcement Learning with Intrinsic Reward Matching [77.34726150561087]
  We present Intrinsic Reward Matching (IRM), which unifies task-agnostic skill pretraining and task-aware finetuning. IRM enables us to utilize pretrained skills far more effectively than previous skill selection methods.
  arXiv  Detail & Related papers  (2022-10-14T00:04:49Z)
• FIRE: A Failure-Adaptive Reinforcement Learning Framework for Edge Computing Migrations [54.34189781923818]
  FIRE is a framework that adapts to rare events by training a RL policy in an edge computing digital twin environment.<n>We propose ImRE, an importance sampling-based Q-learning algorithm, which samples rare events proportionally to their impact on the value function.<n>We show that FIRE reduces costs compared to vanilla RL and the greedy baseline in the event of failures.
  arXiv  Detail & Related papers  (2022-09-28T19:49:39Z)
• Online Auction-Based Incentive Mechanism Design for Horizontal Federated Learning with Budget Constraint [9.503584357135832]
  Federated learning makes it possible for all parties with data isolation to train the model collaboratively and efficiently. To obtain a high-quality model, an incentive mechanism is necessary to motivate more high-quality workers with data and computing power. We propose a reverse auction-based online incentive mechanism for horizontal federated learning with budget constraint.
  arXiv  Detail & Related papers  (2022-01-22T13:37:52Z)
• Harnessing Context for Budget-Limited Crowdsensing with Massive Uncertain Workers [26.835745787064337]
  We propose a Context-Aware Worker Selection (CAWS) algorithm in this paper. CAWS aims at maximizing the expected total sensing revenue efficiently with both budget constraint and capacity constraints respected.
  arXiv  Detail & Related papers  (2021-07-03T09:09:07Z)
• Softmax with Regularization: Better Value Estimation in Multi-Agent Reinforcement Learning [72.28520951105207]
  Overestimation in $Q$-learning is an important problem that has been extensively studied in single-agent reinforcement learning. We propose a novel regularization-based update scheme that penalizes large joint action-values deviating from a baseline. We show that our method provides a consistent performance improvement on a set of challenging StarCraft II micromanagement tasks.
  arXiv  Detail & Related papers  (2021-03-22T14:18:39Z)
• Reinforcement Learning with Efficient Active Feature Acquisition [59.91808801541007]
  In real-life, information acquisition might correspond to performing a medical test on a patient. We propose a model-based reinforcement learning framework that learns an active feature acquisition policy. Key to the success is a novel sequential variational auto-encoder that learns high-quality representations from partially observed states.
  arXiv  Detail & Related papers  (2020-11-02T08:46:27Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.