SkillMimic: Learning Reusable Basketball Skills from Demonstrations

• URL: http://arxiv.org/abs/2408.15270v1
• Date: Mon, 12 Aug 2024 15:19:04 GMT
• Title: SkillMimic: Learning Reusable Basketball Skills from Demonstrations
• Authors: Yinhuai Wang, Qihan Zhao, Runyi Yu, Ailing Zeng, Jing Lin, Zhengyi Luo, Hok Wai Tsui, Jiwen Yu, Xiu Li, Qifeng Chen, Jian Zhang, Lei Zhang, Ping Tan,
• Abstract summary: We propose SkillMimic, a data-driven approach that mimics both human and ball motions to learn a wide variety of basketball skills. SkillMimic employs a unified configuration to learn diverse skills from human-ball motion datasets. The skills acquired by SkillMimic can be easily reused by a high-level controller to accomplish complex basketball tasks.
• Score: 85.23012579911378
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Mastering basketball skills such as diverse layups and dribbling involves complex interactions with the ball and requires real-time adjustments. Traditional reinforcement learning methods for interaction skills rely on labor-intensive, manually designed rewards that do not generalize well across different skills. Inspired by how humans learn from demonstrations, we propose SkillMimic, a data-driven approach that mimics both human and ball motions to learn a wide variety of basketball skills. SkillMimic employs a unified configuration to learn diverse skills from human-ball motion datasets, with skill diversity and generalization improving as the dataset grows. This approach allows training a single policy to learn multiple skills, enabling smooth skill switching even if these switches are not present in the reference dataset. The skills acquired by SkillMimic can be easily reused by a high-level controller to accomplish complex basketball tasks. To evaluate our approach, we introduce two basketball datasets: one estimated through monocular RGB videos and the other using advanced motion capture equipment, collectively containing about 35 minutes of diverse basketball skills. Experiments show that our method can effectively learn various basketball skills included in the dataset with a unified configuration, including various styles of dribbling, layups, and shooting. Furthermore, by training a high-level controller to reuse the acquired skills, we can achieve complex basketball tasks such as layup scoring, which involves dribbling toward the basket, timing the dribble and layup to score, retrieving the rebound, and repeating the process. The project page and video demonstrations are available at https://ingrid789.github.io/SkillMimic/

Related papers

• ExpertAF: Expert Actionable Feedback from Video [81.46431188306397]
  We introduce a novel method to generate actionable feedback from video of a person doing a physical activity. Our method takes a video demonstration and its accompanying 3D body pose and generates expert commentary. Our method is able to reason across multi-modal input combinations to output full-spectrum, actionable coaching.
  arXiv  Detail & Related papers  (2024-08-01T16:13:07Z)
• Offensive Lineup Analysis in Basketball with Clustering Players Based on Shooting Style and Offensive Role [0.9002260638342727]
  The purpose of this study is to analyze more specifically the impact of playing style compatibility on scoring efficiency, focusing only on offense. This study employs two methods to capture the playing styles of players on offense: shooting style clustering using tracking data, and offensive role clustering based on annotated playtypes and advanced statistics. Based on the lineup information derived from these two clusterings, machine learning models Bayesian models that predict statistics representing scoring efficiency were trained and interpreted.
  arXiv  Detail & Related papers  (2024-03-04T03:09:25Z)
• C$\cdot$ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characters [49.83342243500835]
  We present C$cdot$ASE, an efficient framework that learns conditional Adversarial Skill Embeddings for physics-based characters. C$cdot$ASE divides the heterogeneous skill motions into distinct subsets containing homogeneous samples for training a low-level conditional model. The skill-conditioned imitation learning naturally offers explicit control over the character's skills after training.
  arXiv  Detail & Related papers  (2023-09-20T14:34:45Z)
• RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot [56.130215236125224]
  A key challenge in robotic manipulation in open domains is how to acquire diverse and generalizable skills for robots. Recent research in one-shot imitation learning has shown promise in transferring trained policies to new tasks based on demonstrations. This paper aims to unlock the potential for an agent to generalize to hundreds of real-world skills with multi-modal perception.
  arXiv  Detail & Related papers  (2023-07-02T15:33:31Z)
• ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters [123.88692739360457]
  General-purpose motor skills enable humans to perform complex tasks. These skills also provide powerful priors for guiding their behaviors when learning new tasks. We present a framework for learning versatile and reusable skill embeddings for physically simulated characters.
  arXiv  Detail & Related papers  (2022-05-04T06:13:28Z)
• MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale [103.7609761511652]
  We show how a large-scale collective robotic learning system can acquire a repertoire of behaviors simultaneously. New tasks can be continuously instantiated from previously learned tasks. We train and evaluate our system on a set of 12 real-world tasks with data collected from 7 robots.
  arXiv  Detail & Related papers  (2021-04-16T16:38:02Z)
• Fever Basketball: A Complex, Flexible, and Asynchronized Sports Game Environment for Multi-agent Reinforcement Learning [38.4742699455284]
  We introduce the Fever Basketball game, a novel reinforcement learning environment where agents are trained to play basketball game. It is a complex and challenging environment that supports multiple characters, multiple positions, and both the single-agent and multi-agent player control modes. To better simulate real-world basketball games, the execution time of actions differs among players, which makes Fever Basketball a novel asynchronized environment.
  arXiv  Detail & Related papers  (2020-12-06T07:51:59Z)
• Multi-Modal Trajectory Prediction of NBA Players [14.735704310108101]
  We propose a method that captures the multi-modal behavior of players, where they might consider multiple trajectories and select the most advantageous one. The method is built on an LSTM-based architecture predicting multiple trajectories and their probabilities, trained by a multi-modal loss function. Experiments on large, fine-grained NBA tracking data show that the proposed method outperforms the state-of-the-art.
  arXiv  Detail & Related papers  (2020-08-18T11:35:44Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.