CyboRacket: A Perception-to-Action Framework for Humanoid Racket Sports

• URL: http://arxiv.org/abs/2603.14605v1
• Date: Sun, 15 Mar 2026 21:01:14 GMT
• Title: CyboRacket: A Perception-to-Action Framework for Humanoid Racket Sports
• Abstract summary: CyboRacket is a hierarchical perception-to-action framework for humanoid racket sports.<n>It integrates onboard visual perception, physics-based trajectory prediction, and large-scale pre-trained whole-body control.<n>We present results in a vision-based humanoid tennis-hitting task.
• Score: 12.562008425243578
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Dynamic ball-interaction tasks remain challenging for robots because they require tight perception-action coupling under limited reaction time. This challenge is especially pronounced in humanoid racket sports, where successful interception depends on accurate visual tracking, trajectory prediction, coordinated stepping, and stable whole-body striking. Existing robotic racket-sport systems often rely on external motion capture for state estimation or on task-specific low-level controllers that must be retrained across tasks and platforms. We present CyboRacket, a hierarchical perception-to-action framework for humanoid racket sports that integrates onboard visual perception, physics-based trajectory prediction, and large-scale pre-trained whole-body control. The framework uses onboard cameras to track the incoming object, predicts its future trajectory, and converts the estimated interception state into target end-effector and base-motion commands for whole-body execution by SONIC on the Unitree G1 humanoid robot. We evaluate the proposed framework in a vision-based humanoid tennis-hitting task. Experimental results demonstrate real-time visual tracking, trajectory prediction, and successful striking using purely onboard sensing.

Related papers

• Towards Professional Tennis Styles for Humanoid Robots with Adaptive Motion Planning and Tracking [25.008696122611877]
  We propose AdaPT, an Adaptive Motion Planning and Tracking framework.<n>AdaPT learns professional tennis serving and rally styles directly from broadcast videos.<n>Real-world experiments on the Unitree G1 demonstrate the effectiveness of our adaptation mechanism.
  arXiv  Detail & Related papers  (2026-08-20T14:19:36Z)
• Vision-Based Dribbling for Humanoid Soccer via Privileged Representation Learning [2.1866669929322837]
  Dribbling a soccer ball while evading active opponents requires simultaneous balance, precise ball control, and awareness of a dynamic adversary under onboard sensing and real-time constraints.<n>We propose an integrated approach in which a temporal depth encoder is embedded into a reinforcement learning policy through a task-specific projection layer.<n>We apply this framework to a simulated Booster T1 humanoid robot and show that it is possible to learn vision-based, opponent-aware dribbling directly from depth observations.
  arXiv  Detail & Related papers  (2026-07-14T12:27:58Z)
• ActiveGlasses: Learning Manipulation with Active Vision from Ego-centric Human Demonstration [51.69384671495837]
  ActiveGlasses is a system for learning robot manipulation from ego-centric human demonstrations with active vision.<n>A stereo camera mounted on smart glasses serves as the sole perception device for both data collection and policy inference.<n>To enable zero-transfer, we extract object trajectories from demonstrations and use an object-centric point-cloud policy to jointly predict manipulation and head movement.
  arXiv  Detail & Related papers  (2026-04-09T17:59:08Z)
• SMASH: Mastering Scalable Whole-Body Skills for Humanoid Ping-Pong with Egocentric Vision [53.709632118234936]
  Methodname is a modular system for agile humanoid table tennis.<n>It unifies scalable whole-body skill learning with onboard egocentric perception.<n>We demonstrate the first humanoid table-tennis system capable of consecutive strikes using onboard sensing alone.
  arXiv  Detail & Related papers  (2026-04-01T17:11:52Z)
• OmniTrack: General Motion Tracking via Physics-Consistent Reference [54.256312586493415]
  We introduce OmniTrack, a general tracking framework that explicitly decouples physical feasibility from general motion tracking.<n>Experiments show that OmniTrack improves tracking accuracy and demonstrates strong generalization to unseen motions.<n>In real-world tests, OmniTrack achieves hour-long, consistent, and stable tracking, including complex acrobatic motions such as flips and cartwheels.
  arXiv  Detail & Related papers  (2026-02-27T09:12:15Z)
• TeleGate: Whole-Body Humanoid Teleoperation via Gated Expert Selection with Motion Prior [27.930225667015872]
  This paper presents TeleGate, a unified whole-body teleoperation framework for humanoid robots.<n>Our key idea is to preserve the full capability of domain-specific expert policies by training a lightweight gating network.<n>To compensate for the absence of future reference trajectories in real-time teleoperation, we introduce a VAE-based motion prior module.
  arXiv  Detail & Related papers  (2026-02-10T10:14:06Z)
• Learning Soccer Skills for Humanoid Robots: A Progressive Perception-Action Framework [50.6604971348824]
  Soccer presents a significant challenge for humanoid robots, demanding tightly integrated perception-action capabilities.<n>We propose Perception-Action integrated Decision-making (PAiD), a progressive architecture that decomposes soccer skill acquisition into three stages.<n> Experiments on the Unitree G1 demonstrate high-fidelity human-like kicking with robust performance under diverse conditions.
  arXiv  Detail & Related papers  (2026-02-05T05:05:03Z)
• Physical Autoregressive Model for Robotic Manipulation without Action Pretraining [65.8971623698511]
  We build upon autoregressive video generation models to propose a Physical Autoregressive Model (PAR)<n>PAR leverages the world knowledge embedded in video pretraining to understand physical dynamics without requiring action pretraining.<n>Experiments on the ManiSkill benchmark show that PAR achieves a 100% success rate on the PushCube task.
  arXiv  Detail & Related papers  (2025-08-13T13:54:51Z)
• Light Future: Multimodal Action Frame Prediction via InstructPix2Pix [0.764671395172401]
  This paper proposes a novel, efficient, and lightweight approach for robot action prediction.<n>It offers significantly reduced computational cost and inference latency compared to conventional video prediction models.<n>It pioneers the adaptation of the InstructPix2Pix model for forecasting future visual frames in robotic tasks.
  arXiv  Detail & Related papers  (2025-07-20T03:57:18Z)
• KungfuBot: Physics-Based Humanoid Whole-Body Control for Learning Highly-Dynamic Skills [58.73043119128804]
  This paper presents a physics-based humanoid control framework, aiming to master highly-dynamic human behaviors such as Kungfu and dancing.<n>For motion processing, we design a pipeline to extract, filter out, correct, and retarget motions, while ensuring compliance with physical constraints.<n>For motion imitation, we formulate a bi-level optimization problem to dynamically adjust the tracking accuracy tolerance.<n>In experiments, we train whole-body control policies to imitate a set of highly-dynamic motions.
  arXiv  Detail & Related papers  (2025-06-15T13:58:53Z)
• Learning coordinated badminton skills for legged manipulators [12.514822343998908]
  We introduce an approach for enabling legged mobile manipulators to play badminton.<n>We propose a unified reinforcement learning-based control policy for whole-body visuomotor skills.<n>Our method includes a shuttlecock prediction model, constrained reinforcement learning for robust motion control, and integrated system identification techniques.
  arXiv  Detail & Related papers  (2025-05-29T01:26:30Z)
• Open-World Drone Active Tracking with Goal-Centered Rewards [62.21394499788672]
  Drone Visual Active Tracking aims to autonomously follow a target object by controlling the motion system based on visual observations.<n>We propose DAT, the first open-world drone active air-to-ground tracking benchmark.<n>We also propose GC-VAT, which aims to improve the performance of drone tracking targets in complex scenarios.
  arXiv  Detail & Related papers  (2024-12-01T09:37:46Z)
• Track2Act: Predicting Point Tracks from Internet Videos enables Generalizable Robot Manipulation [65.46610405509338]
  We seek to learn a generalizable goal-conditioned policy that enables zero-shot robot manipulation. Our framework,Track2Act predicts tracks of how points in an image should move in future time-steps based on a goal. We show that this approach of combining scalably learned track prediction with a residual policy enables diverse generalizable robot manipulation.
  arXiv  Detail & Related papers  (2024-05-02T17:56:55Z)
• PUCK: Parallel Surface and Convolution-kernel Tracking for Event-Based Cameras [4.110120522045467]
  Event-cameras can guarantee fast visual sensing in dynamic environments, but require a tracking algorithm that can keep up with the high data rate induced by the robot ego-motion. We introduce a novel tracking method that leverages the Exponential Reduced Ordinal Surface (EROS) data representation to decouple event-by-event processing and tracking. We propose the task of tracking the air hockey puck sliding on a surface, with the future aim of controlling the iCub robot to reach the target precisely and on time.
  arXiv  Detail & Related papers  (2022-05-16T13:23:52Z)
• TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wild [77.59069361196404]
  TRiPOD is a novel method for predicting body dynamics based on graph attentional networks. To incorporate a real-world challenge, we learn an indicator representing whether an estimated body joint is visible/invisible at each frame. Our evaluation shows that TRiPOD outperforms all prior work and state-of-the-art specifically designed for each of the trajectory and pose forecasting tasks.
  arXiv  Detail & Related papers  (2021-04-08T20:01:00Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.