Human-Like Goalkeeping in a Realistic Football Simulation: a Sample-Efficient Reinforcement Learning Approach

• URL: http://arxiv.org/abs/2510.23216v3
• Date: Thu, 30 Oct 2025 14:45:38 GMT
• Title: Human-Like Goalkeeping in a Realistic Football Simulation: a Sample-Efficient Reinforcement Learning Approach
• Authors: Alessandro Sestini, Joakim Bergdahl, Jean-Philippe Barrette-LaPierre, Florian Fuchs, Brady Chen, Michael Jones, Linus Gisslén,
• Abstract summary: This paper proposes a sample-efficient Deep Reinforcement Learning (DRL) method tailored for training and fine-tuning agents in industrial settings.<n>We evaluate our method training a goalkeeper agent in EA SPORTS FC 25, one of the best-selling football simulations today.<n>Our agent outperforms the game's built-in AI by 10% in ball saving rate.
• Score: 35.515515697546554
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: While several high profile video games have served as testbeds for Deep Reinforcement Learning (DRL), this technique has rarely been employed by the game industry for crafting authentic AI behaviors. Previous research focuses on training super-human agents with large models, which is impractical for game studios with limited resources aiming for human-like agents. This paper proposes a sample-efficient DRL method tailored for training and fine-tuning agents in industrial settings such as the video game industry. Our method improves sample efficiency of value-based DRL by leveraging pre-collected data and increasing network plasticity. We evaluate our method training a goalkeeper agent in EA SPORTS FC 25, one of the best-selling football simulations today. Our agent outperforms the game's built-in AI by 10% in ball saving rate. Ablation studies show that our method trains agents 50% faster compared to standard DRL methods. Finally, qualitative evaluation from domain experts indicates that our approach creates more human-like gameplay compared to hand-crafted agents. As a testament to the impact of the approach, the method has been adopted for use in the most recent release of the series.

Related papers

• Multimodal Reinforcement Learning with Agentic Verifier for AI Agents [131.46008226323423]
  Argos is a principled multimodal reward agent to train reasoning models for agentic tasks.<n>By leveraging our agentic verifier across both SFT data and RL training, our model achieves state-of-the-art results.
  arXiv  Detail & Related papers  (2025-12-03T04:42:47Z)
• Reinforcing Competitive Multi-Agents for Playing 'So Long Sucker' [0.12234742322758417]
  This paper investigates the strategy game So Long Sucker (SLS) as a novel benchmark for multi-agent reinforcement learning (MARL)<n>We introduce the first publicly available computational framework for SLS, complete with a graphical user interface and benchmarking support for reinforcement learning algorithms.
  arXiv  Detail & Related papers  (2024-11-17T12:38:13Z)
• Scaling Offline Model-Based RL via Jointly-Optimized World-Action Model Pretraining [49.730897226510095]
  We introduce JOWA: Jointly-Reinforced World-Action model, an offline model-based RL agent pretrained on Atari games with 6 billion tokens data.<n>Our largest agent, with 150 million parameters, 78.9% human-level performance on pretrained games using only 10% subsampled offline data, outperforming existing state-of-the-art large-scale offline RL baselines by 31.6% on averange.
  arXiv  Detail & Related papers  (2024-10-01T10:25:03Z)
• An Extremely Data-efficient and Generative LLM-based Reinforcement Learning Agent for Recommenders [1.0154385852423122]
  reinforcement learning (RL) algorithms have been instrumental in maximizing long-term customer satisfaction and avoiding short-term, myopic goals in industrial recommender systems. The goal is to train an RL agent to maximize the purchase reward given a detailed human instruction describing a desired product. This report also evaluates the RL agents trained using generative trajectories.
  arXiv  Detail & Related papers  (2024-08-28T10:31:50Z)
• DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning [61.10299147201369]
  This paper introduces a novel autonomous RL approach, called DigiRL, for training in-the-wild device control agents. We build a scalable and parallelizable Android learning environment equipped with a VLM-based evaluator. We demonstrate the effectiveness of DigiRL using the Android-in-the-Wild dataset, where our 1.3B VLM trained with RL achieves a 49.5% absolute improvement.
  arXiv  Detail & Related papers  (2024-06-14T17:49:55Z)
• Behavioural Cloning in VizDoom [1.4999444543328293]
  This paper describes methods for training autonomous agents to play the game "Doom 2" through Imitation Learning (IL) We also explore how Reinforcement Learning (RL) compares to IL for humanness by comparing camera movement and trajectory data.
  arXiv  Detail & Related papers  (2024-01-08T16:15:43Z)
• Minimax Exploiter: A Data Efficient Approach for Competitive Self-Play [12.754819077905061]
  Minimax Exploiter is a game theoretic approach to exploiting Main Agents that leverages knowledge of its opponents. We validate our approach in a diversity of settings, including simple turn based games, the arcade learning environment, and For Honor, a modern video game.
  arXiv  Detail & Related papers  (2023-11-28T19:34:40Z)
• TiKick: Toward Playing Multi-agent Football Full Games from Single-agent Demonstrations [31.596018856092513]
  Tikick is the first learning-based AI system that can take over the multi-agent Google Research Football full game. To the best of our knowledge, Tikick is the first learning-based AI system that can take over the multi-agent Google Research Football full game.
  arXiv  Detail & Related papers  (2021-10-09T08:34:58Z)
• Explaining Deep Reinforcement Learning Agents In The Atari Domain through a Surrogate Model [78.69367679848632]
  We describe a lightweight and effective method to derive explanations for deep RL agents. Our method relies on a transformation of the pixel-based input of the RL agent to an interpretable, percept-like input representation. We then train a surrogate model, which is itself interpretable, to replicate the behavior of the target, deep RL agent.
  arXiv  Detail & Related papers  (2021-10-07T05:01:44Z)
• Predicting Game Engagement and Difficulty Using AI Players [3.0501851690100277]
  This paper presents a novel approach to automated playtesting for the prediction of human player behavior and experience. It has previously been demonstrated that Deep Reinforcement Learning game-playing agents can predict both game difficulty and player engagement. We improve this approach by enhancing DRL with Monte Carlo Tree Search (MCTS)
  arXiv  Detail & Related papers  (2021-07-26T09:31:57Z)
• Learning to Run with Potential-Based Reward Shaping and Demonstrations from Video Data [70.540936204654]
  "Learning to run" competition was to train a two-legged model of a humanoid body to run in a simulated race course with maximum speed. All submissions took a tabula rasa approach to reinforcement learning (RL) and were able to produce relatively fast, but not optimal running behaviour. We demonstrate how data from videos of human running can be used to shape the reward of the humanoid learning agent.
  arXiv  Detail & Related papers  (2020-12-16T09:46:58Z)
• Agent57: Outperforming the Atari Human Benchmark [15.75730239983062]
  Atari games have been a long-standing benchmark in reinforcement learning. We propose Agent57, the first deep RL agent that outperforms the standard human benchmark on all 57 Atari games.
  arXiv  Detail & Related papers  (2020-03-30T11:33:16Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.