Learning Tennis Strategy Through Curriculum-Based Dueling Double Deep Q-Networks

• URL: http://arxiv.org/abs/2512.22186v1
• Date: Sat, 20 Dec 2025 04:22:07 GMT
• Title: Learning Tennis Strategy Through Curriculum-Based Dueling Double Deep Q-Networks
• Abstract summary: Tennis strategy optimization is a challenging sequential decision-making problem involving hierarchical scoring, outcomes, long-horizon credit assignment, physical fatigue, and adaptation to opponent skill.<n>I present a reinforcement learning framework that integrates a custom tennis simulation environment with a Dueling Double Deep Q-Network trained using curriculum learning.
• Score: 0.3181222613501246
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Tennis strategy optimization is a challenging sequential decision-making problem involving hierarchical scoring, stochastic outcomes, long-horizon credit assignment, physical fatigue, and adaptation to opponent skill. I present a reinforcement learning framework that integrates a custom tennis simulation environment with a Dueling Double Deep Q-Network(DDQN) trained using curriculum learning. The environment models complete tennis scoring at the level of points, games, and sets, rally-level tactical decisions across ten discrete action categories, symmetric fatigue dynamics, and a continuous opponent skill parameter. The dueling architecture decomposes action-value estimation into state-value and advantage components, while double Q-learning reduces overestimation bias and improves training stability in this long-horizon stochastic domain. Curriculum learning progressively increases opponent difficulty from 0.40 to 0.50, enabling robust skill acquisition without the training collapse observed under fixed opponents. Across extensive evaluations, the trained agent achieves win rates between 98 and 100 percent against balanced opponents and maintains strong performance against more challenging opponents. Serve efficiency ranges from 63.0 to 67.5 percent, and return efficiency ranges from 52.8 to 57.1 percent. Ablation studies demonstrate that both the dueling architecture and curriculum learning are necessary for stable convergence, while a standard DQN baseline fails to learn effective policies. Despite strong performance, tactical analysis reveals a pronounced defensive bias, with the learned policy prioritizing error avoidance and prolonged rallies over aggressive point construction. These results highlight a limitation of win-rate driven optimization in simplified sports simulations and emphasize the importance of reward design for realistic sports reinforcement learning.

Related papers

• Reproducing AlphaZero on Tablut: Self-Play RL for an Asymmetric Board Game [0.6574517227976925]
  This work investigates the adaptation of the AlphaZero reinforcement learning algorithm to Tablut, an asymmetric historical board game.<n>Over 100 self-play iterations, the modified model demonstrated steady improvement, achieving a BayesElo rating of 1235.<n>Ultimately, the experiments confirm that AlphaZero's self-play framework can transfer to highly asymmetric games.
  arXiv  Detail & Related papers  (2026-04-07T06:16:19Z)
• Actor-Curator: Co-adaptive Curriculum Learning via Policy-Improvement Bandits for RL Post-Training [63.34044358216334]
  ACTOR-CURATOR is a scalable and fully automated curriculum learning framework for reinforcement learning post-training of large language models.<n> Empirically, ACTOR-CURATOR consistently outperforms uniform sampling and strong curriculum baselines.
  arXiv  Detail & Related papers  (2026-02-24T04:19:48Z)
• DiffFP: Learning Behaviors from Scratch via Diffusion-based Fictitious Play [5.8808473430456525]
  We propose DiffFP, a fictitious play framework that estimates the best response to unseen opponents in zero-sum games.<n>We validate our method on complex multi-agent environments, including racing and multi-particle zero-sum games.<n>Our approach achieves up to 3$times$ faster convergence and 30$times$ higher success rates on average against RL-based baselines.
  arXiv  Detail & Related papers  (2025-11-17T09:48:29Z)
• Enhancing Language Agent Strategic Reasoning through Self-Play in Adversarial Games [60.213483076150844]
  We propose a Step-level poliCy Optimization method through Play-And-Learn, SCO-PAL.<n>We conduct a detailed analysis of opponent selection by setting opponents at different levels and find that self-play is the most effective way to improve strategic reasoning.<n>We achieve a 54.76% win rate against GPT-4 in six adversarial games.
  arXiv  Detail & Related papers  (2025-10-19T09:03:28Z)
• SPIRAL: Self-Play on Zero-Sum Games Incentivizes Reasoning via Multi-Agent Multi-Turn Reinforcement Learning [27.20778530252474]
  SPIRAL is a self-play framework where models learn by playing multi-turn, zero-sum games against continuously improving versions of themselves.<n>Using SPIRAL, self-play on zero-sum games produces reasoning capabilities that transfer broadly.<n>Analysis reveals that this transfer occurs through three cognitive patterns: systematic decomposition, expected value calculation, and case-by-case analysis.
  arXiv  Detail & Related papers  (2025-06-30T17:58:13Z)
• QFAL: Quantum Federated Adversarial Learning [3.7329133349509926]
  Quantum federated learning (QFL) merges the privacy advantages of federated systems with the computational potential of quantum neural networks (QNNs)<n>This work pioneers the integration of adversarial training into QFL, proposing a robust framework, quantum federated adversarial learning (QFAL)<n>We evaluate the interplay between three critical factors: client count (5, 10, 15), adversarial training coverage (0-100%), and adversarial attack perturbation strength (epsilon = 0.01-0.5)<n>We conclude that a carefully chosen combination of client count and adversarial coverage is critical for mitigating adversarial vulnerabilities in QFL.
  arXiv  Detail & Related papers  (2025-02-28T15:54:39Z)
• Enhancing Adversarial Training via Reweighting Optimization Trajectory [72.75558017802788]
  A number of approaches have been proposed to address drawbacks such as extra regularization, adversarial weights, and training with more data. We propose a new method named textbfWeighted Optimization Trajectories (WOT) that leverages the optimization trajectories of adversarial training in time. Our results show that WOT integrates seamlessly with the existing adversarial training methods and consistently overcomes the robust overfitting issue.
  arXiv  Detail & Related papers  (2023-06-25T15:53:31Z)
• Strength-Adaptive Adversarial Training [103.28849734224235]
  Adversarial training (AT) is proven to reliably improve network's robustness against adversarial data. Current AT with a pre-specified perturbation budget has limitations in learning a robust network. We propose emphStrength-Adaptive Adversarial Training (SAAT) to overcome these limitations.
  arXiv  Detail & Related papers  (2022-10-04T00:22:37Z)
• Enhancing Adversarial Training with Feature Separability [52.39305978984573]
  We introduce a new concept of adversarial training graph (ATG) with which the proposed adversarial training with feature separability (ATFS) enables to boost the intra-class feature similarity and increase inter-class feature variance. Through comprehensive experiments, we demonstrate that the proposed ATFS framework significantly improves both clean and robust performance.
  arXiv  Detail & Related papers  (2022-05-02T04:04:23Z)
• Enhancing Adversarial Robustness for Deep Metric Learning [77.75152218980605]
  adversarial robustness of deep metric learning models has to be improved. In order to avoid model collapse due to excessively hard examples, the existing defenses dismiss the min-max adversarial training. We propose Hardness Manipulation to efficiently perturb the training triplet till a specified level of hardness for adversarial training.
  arXiv  Detail & Related papers  (2022-03-02T22:27:44Z)
• Improving the affordability of robustness training for DNNs [11.971637253035107]
  We show that the initial phase of adversarial training is redundant and can be replaced with natural training which significantly improves the computational efficiency. We show that our proposed method can reduce the training time by a factor of up to 2.5 with comparable or better model test accuracy and generalization on various strengths of adversarial attacks.
  arXiv  Detail & Related papers  (2020-02-11T07:29:45Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.