SkillMimic: Learning Basketball Interaction Skills from Demonstrations

• URL: http://arxiv.org/abs/2408.15270v2
• Date: Fri, 28 Mar 2025 08:56:06 GMT
• Title: SkillMimic: Learning Basketball Interaction Skills from Demonstrations
• Authors: Yinhuai Wang, Qihan Zhao, Runyi Yu, Hok Wai Tsui, Ailing Zeng, Jing Lin, Zhengyi Luo, Jiwen Yu, Xiu Li, Qifeng Chen, Jian Zhang, Lei Zhang, Ping Tan,
• Abstract summary: We introduce SkillMimic, a unified data-driven framework that fundamentally changes how agents learn interaction skills.<n>Our key insight is that a unified HOI imitation reward can effectively capture the essence of diverse interaction patterns from HOI datasets.<n>For evaluation, we collect and introduce two basketball datasets containing approximately 35 minutes of diverse basketball skills.
• Score: 85.23012579911378
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Traditional reinforcement learning methods for human-object interaction (HOI) rely on labor-intensive, manually designed skill rewards that do not generalize well across different interactions. We introduce SkillMimic, a unified data-driven framework that fundamentally changes how agents learn interaction skills by eliminating the need for skill-specific rewards. Our key insight is that a unified HOI imitation reward can effectively capture the essence of diverse interaction patterns from HOI datasets. This enables SkillMimic to learn a single policy that not only masters multiple interaction skills but also facilitates skill transitions, with both diversity and generalization improving as the HOI dataset grows. For evaluation, we collect and introduce two basketball datasets containing approximately 35 minutes of diverse basketball skills. Extensive experiments show that SkillMimic successfully masters a wide range of basketball skills including stylistic variations in dribbling, layup, and shooting. Moreover, these learned skills can be effectively composed by a high-level controller to accomplish complex and long-horizon tasks such as consecutive scoring, opening new possibilities for scalable and generalizable interaction skill learning. Project page: https://ingrid789.github.io/SkillMimic/

Related papers

• Pretrained Bayesian Non-parametric Knowledge Prior in Robotic Long-Horizon Reinforcement Learning [10.598207472087578]
  Reinforcement learning (RL) methods typically learn new tasks from scratch, often disregarding prior knowledge that could accelerate the learning process. This work introduces a method that models potential primitive skill motions as having non-parametric properties with an unknown number of underlying features. We utilize a non-parametric model, specifically Dirichlet Process Mixtures, enhanced with birth and merge, to pre-train a skill prior that effectively captures the diverse nature of skills.
  arXiv  Detail & Related papers  (2025-03-27T20:43:36Z)
• ExpertAF: Expert Actionable Feedback from Video [81.46431188306397]
  We introduce a novel method to generate actionable feedback from video of a person doing a physical activity. Our method takes a video demonstration and its accompanying 3D body pose and generates expert commentary. Our method is able to reason across multi-modal input combinations to output full-spectrum, actionable coaching.
  arXiv  Detail & Related papers  (2024-08-01T16:13:07Z)
• Offensive Lineup Analysis in Basketball with Clustering Players Based on Shooting Style and Offensive Role [0.9002260638342727]
  The purpose of this study is to analyze more specifically the impact of playing style compatibility on scoring efficiency, focusing only on offense. This study employs two methods to capture the playing styles of players on offense: shooting style clustering using tracking data, and offensive role clustering based on annotated playtypes and advanced statistics. Based on the lineup information derived from these two clusterings, machine learning models Bayesian models that predict statistics representing scoring efficiency were trained and interpreted.
  arXiv  Detail & Related papers  (2024-03-04T03:09:25Z)
• C$\cdot$ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characters [49.83342243500835]
  We present C$cdot$ASE, an efficient framework that learns conditional Adversarial Skill Embeddings for physics-based characters. C$cdot$ASE divides the heterogeneous skill motions into distinct subsets containing homogeneous samples for training a low-level conditional model. The skill-conditioned imitation learning naturally offers explicit control over the character's skills after training.
  arXiv  Detail & Related papers  (2023-09-20T14:34:45Z)
• RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot [56.130215236125224]
  A key challenge in robotic manipulation in open domains is how to acquire diverse and generalizable skills for robots. Recent research in one-shot imitation learning has shown promise in transferring trained policies to new tasks based on demonstrations. This paper aims to unlock the potential for an agent to generalize to hundreds of real-world skills with multi-modal perception.
  arXiv  Detail & Related papers  (2023-07-02T15:33:31Z)
• Behavior Contrastive Learning for Unsupervised Skill Discovery [75.6190748711826]
  We propose a novel unsupervised skill discovery method through contrastive learning among behaviors. Under mild assumptions, our objective maximizes the MI between different behaviors based on the same skill. Our method implicitly increases the state entropy to obtain better state coverage.
  arXiv  Detail & Related papers  (2023-05-08T06:02:11Z)
• Skill Reinforcement Learning and Planning for Open-World Long-Horizon Tasks [31.084848672383185]
  We study building multi-task agents in open-world environments. We convert the multi-task learning problem into learning basic skills and planning over the skills. Our method accomplishes 40 diverse Minecraft tasks, where many tasks require sequentially executing for more than 10 skills.
  arXiv  Detail & Related papers  (2023-03-29T09:45:50Z)
• Choreographer: Learning and Adapting Skills in Imagination [60.09911483010824]
  We present Choreographer, a model-based agent that exploits its world model to learn and adapt skills in imagination. Our method decouples the exploration and skill learning processes, being able to discover skills in the latent state space of the model. Choreographer is able to learn skills both from offline data, and by collecting data simultaneously with an exploration policy.
  arXiv  Detail & Related papers  (2022-11-23T23:31:14Z)
• ASE: Large-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters [123.88692739360457]
  General-purpose motor skills enable humans to perform complex tasks. These skills also provide powerful priors for guiding their behaviors when learning new tasks. We present a framework for learning versatile and reusable skill embeddings for physically simulated characters.
  arXiv  Detail & Related papers  (2022-05-04T06:13:28Z)
• Discovering Generalizable Skills via Automated Generation of Diverse Tasks [82.16392072211337]
  We propose a method to discover generalizable skills via automated generation of a diverse set of tasks. As opposed to prior work on unsupervised discovery of skills, our method pairs each skill with a unique task produced by a trainable task generator. A task discriminator defined on the robot behaviors in the generated tasks is jointly trained to estimate the evidence lower bound of the diversity objective. The learned skills can then be composed in a hierarchical reinforcement learning algorithm to solve unseen target tasks.
  arXiv  Detail & Related papers  (2021-06-26T03:41:51Z)
• MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale [103.7609761511652]
  We show how a large-scale collective robotic learning system can acquire a repertoire of behaviors simultaneously. New tasks can be continuously instantiated from previously learned tasks. We train and evaluate our system on a set of 12 real-world tasks with data collected from 7 robots.
  arXiv  Detail & Related papers  (2021-04-16T16:38:02Z)
• Skillearn: Machine Learning Inspired by Humans' Learning Skills [15.125072827275766]
  We are interested in investigating whether humans' learning skills can be borrowed to help machines learn better. Specifically, we aim to formalize these skills and leverage them to train better machine learning (ML) models. To achieve this goal, we develop a general framework -- Skillearn, which provides a principled way to represent humans' learning skills mathematically. In two case studies, we apply Skillearn to formalize two learning skills of humans: learning by passing tests and interleaving learning, and use the formalized skills to improve neural architecture search.
  arXiv  Detail & Related papers  (2020-12-09T04:56:22Z)
• Fever Basketball: A Complex, Flexible, and Asynchronized Sports Game Environment for Multi-agent Reinforcement Learning [38.4742699455284]
  We introduce the Fever Basketball game, a novel reinforcement learning environment where agents are trained to play basketball game. It is a complex and challenging environment that supports multiple characters, multiple positions, and both the single-agent and multi-agent player control modes. To better simulate real-world basketball games, the execution time of actions differs among players, which makes Fever Basketball a novel asynchronized environment.
  arXiv  Detail & Related papers  (2020-12-06T07:51:59Z)
• Accelerating Reinforcement Learning with Learned Skill Priors [20.268358783821487]
  Most modern reinforcement learning approaches learn every task from scratch. One approach for leveraging prior knowledge is to transfer skills learned on prior tasks to the new task. We show that learned skill priors are essential for effective skill transfer from rich datasets.
  arXiv  Detail & Related papers  (2020-10-22T17:59:51Z)
• Multi-Modal Trajectory Prediction of NBA Players [14.735704310108101]
  We propose a method that captures the multi-modal behavior of players, where they might consider multiple trajectories and select the most advantageous one. The method is built on an LSTM-based architecture predicting multiple trajectories and their probabilities, trained by a multi-modal loss function. Experiments on large, fine-grained NBA tracking data show that the proposed method outperforms the state-of-the-art.
  arXiv  Detail & Related papers  (2020-08-18T11:35:44Z)
• ELSIM: End-to-end learning of reusable skills through intrinsic motivation [0.0]
  We present a novel reinforcement learning architecture which hierarchically learns and represents self-generated skills in an end-to-end way. With this architecture, an agent focuses only on task-rewarded skills while keeping the learning process of skills bottom-up.
  arXiv  Detail & Related papers  (2020-06-23T11:20:46Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.