Expressive Robotic Pianist: Mastering Complex Piano Repertoire with Graph-Mimic and Musical Dynamics

• URL: http://arxiv.org/abs/2609.10844v1
• Date: Wed, 09 Sep 2026 21:26:29 GMT
• Title: Expressive Robotic Pianist: Mastering Complex Piano Repertoire with Graph-Mimic and Musical Dynamics
• Abstract summary: We show that a dexterous robotic hand can achieve high-fidelity performance across a diverse piano repertoire.<n>Our approach provides a robust pathway for robotic systems to move beyond mere mechanical accuracy.
• Score: 5.506290956393904
• License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
• Abstract: Enabling robots to perform musical instruments with human-level expressivity represents a frontier in bridging the gap between mechanical precision and artistic interpretation. Despite advances in robotic dexterity, replicating the fluid finger transitions and nuanced dynamic control characteristic of human pianists remains a significant challenge. Through a reinforcement learning-based control framework, we demonstrate that a dexterous robotic hand can achieve high-fidelity performance across a diverse piano repertoire. Central to our approach is a graph-based optimization strategy that guides the robot to generate natural pre-press and key-press fingering strategies that closely resemble human movement patterns. To achieve expressive sound production, the control system is coupled with a physics-inspired acoustic model that modulates keypress velocity to accurately reproduce the dynamic variations specified in musical scores. Quantitative evaluations demonstrate that our expressive control model significantly outperforms baseline methods in both finger morphology similarity and dynamic velocity accuracy. In a perceptual test involving participants from diverse listener groups, performances generated by our system are significantly preferred over baseline robotic performances and are indistinguishable from human performances for non-professional audiences. Furthermore, extensive experiments across multiple musical styles confirm that our method maintains high note-level accuracy while achieving expressive performance. Our approach provides a robust pathway for robotic systems to move beyond mere mechanical accuracy, elevating robotic musicianship to a level of expressive performance comparable to human pianists.

Related papers

• Humanoid Musical Robots as Experimental Interfaces for Music-Evoked Emotion [1.3750624267664158]
  We argue that musical humanoid robots are experimental interfaces for investigating music-evoked emotions.<n>We show that humanoid robots are well-suited as they enable parametric control of performance variables.<n>Our work positions musical humanoid robots as a methodological platform that enables future controlled investigations of music-evoked emotions.
  arXiv  Detail & Related papers  (2026-08-20T03:23:36Z)
• Co-policy: Responsive Human-Robot Co-Creation for Musical Performances [58.92814200971293]
  Co-policy is a framework for human-robot musical co-creation.<n>It separates semantic intent grounding, constrained musical variation, and visuomotor execution.<n>We show improved intent alignment, execution accuracy, and response frequency over diffusion-policy and ablated baselines.
  arXiv  Detail & Related papers  (2026-06-18T08:08:16Z)
• MUSIC: Learning Muscle-Driven Dexterous Hand Control [22.43764880740859]
  We present a data-driven approach for physics-based, muscle-driven dexterous control.<n>Our approach combines high-frequency muscle-level control with low-frequency latent-space coordination.<n>We show that our approach can synthesize coordinated bimanual motions with accurate key presses.
  arXiv  Detail & Related papers  (2026-04-26T21:05:34Z)
• Aggregating swarms through morphology handling design contingencies: from the sweet spot to a rich expressivity [0.12744523252873352]
  We build on a faithful physical model of the self-aligning dynamics of the robots.<n>We show that exploring a range of self-alignment strength allows for a rich expressivity of collective behaviors.
  arXiv  Detail & Related papers  (2026-01-12T14:58:47Z)
• From Score to Sound: An End-to-End MIDI-to-Motion Pipeline for Robotic Cello Performance [1.6971047370270889]
  We propose a novel end-to-end MIDI score to robotic motion pipeline.<n>This pipeline converts musical input directly into collision-aware bowing motions for a UR5e robot cellist.<n>To demonstrate the effectiveness of our method in comparison to human performers, we introduce the Musical Turing Test.
  arXiv  Detail & Related papers  (2026-01-07T04:11:00Z)
• The Ghost in the Keys: A Disklavier Demo for Human-AI Musical Co-Creativity [59.78509280246215]
  Aria-Duet is an interactive system facilitating a real-time musical duet between a human pianist and Aria, a state-of-the-art generative model.<n>We analyze the system's output from a musicological perspective, finding the model can maintain stylistic semantics and develop coherent phrasal ideas.
  arXiv  Detail & Related papers  (2025-11-03T15:26:01Z)
• FreeAction: Training-Free Techniques for Enhanced Fidelity of Trajectory-to-Video Generation [50.39748673817223]
  We introduce two training-free, inference-time techniques that fully exploit explicit action parameters in robot video generation.<n>First, action-scaled classifier-free guidance dynamically modulates guidance strength in proportion to action magnitude, enhancing controllability over motion intensity.<n>Second, action-scaled noise truncation adjusts the distribution of initially sampled noise to better align with the desired motion dynamics.
  arXiv  Detail & Related papers  (2025-09-29T03:30:40Z)
• Physical Autoregressive Model for Robotic Manipulation without Action Pretraining [65.8971623698511]
  We build upon autoregressive video generation models to propose a Physical Autoregressive Model (PAR)<n>PAR leverages the world knowledge embedded in video pretraining to understand physical dynamics without requiring action pretraining.<n>Experiments on the ManiSkill benchmark show that PAR achieves a 100% success rate on the PushCube task.
  arXiv  Detail & Related papers  (2025-08-13T13:54:51Z)
• RP1M: A Large-Scale Motion Dataset for Piano Playing with Bi-Manual Dexterous Robot Hands [57.64308229980045]
  We introduce the Robot Piano 1 Million dataset, containing bi-manual robot piano playing motion data of more than one million trajectories. We formulate finger placements as an optimal transport problem, thus, enabling automatic annotation of vast amounts of unlabeled songs. Benchmarking existing imitation learning approaches shows that such approaches reach state-of-the-art robot piano playing performance by leveraging RP1M.
  arXiv  Detail & Related papers  (2024-08-20T17:56:52Z)
• DiffuseBot: Breeding Soft Robots With Physics-Augmented Generative Diffusion Models [102.13968267347553]
  We present DiffuseBot, a physics-augmented diffusion model that generates soft robot morphologies capable of excelling in a wide spectrum of tasks. We showcase a range of simulated and fabricated robots along with their capabilities.
  arXiv  Detail & Related papers  (2023-11-28T18:58:48Z)
• Reconstructing Human Expressiveness in Piano Performances with a Transformer Network [1.5883812630616518]
  We propose a novel approach for reconstructing human expressiveness in piano performance with a multi-layer bi-directional Transformer encoder. To address the needs for accurately captured and score-aligned performance data in training neural networks, we use transcribed scores obtained from an existing transcription model to train our model.
  arXiv  Detail & Related papers  (2023-06-09T17:05:53Z)
• RoboPianist: Dexterous Piano Playing with Deep Reinforcement Learning [61.10744686260994]
  We introduce RoboPianist, a system that enables simulated anthropomorphic hands to learn an extensive repertoire of 150 piano pieces. We additionally introduce an open-sourced environment, benchmark of tasks, interpretable evaluation metrics, and open challenges for future study.
  arXiv  Detail & Related papers  (2023-04-09T03:53:05Z)
• Data-driven emotional body language generation for social robotics [58.88028813371423]
  In social robotics, endowing humanoid robots with the ability to generate bodily expressions of affect can improve human-robot interaction and collaboration. We implement a deep learning data-driven framework that learns from a few hand-designed robotic bodily expressions. The evaluation study found that the anthropomorphism and animacy of the generated expressions are not perceived differently from the hand-designed ones.
  arXiv  Detail & Related papers  (2022-05-02T09:21:39Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.