SteadyTray: Learning Object Balancing Tasks in Humanoid Tray Transport via Residual Reinforcement Learning

• URL: http://arxiv.org/abs/2603.10306v1
• Date: Wed, 11 Mar 2026 01:05:46 GMT
• Title: SteadyTray: Learning Object Balancing Tasks in Humanoid Tray Transport via Residual Reinforcement Learning
• Abstract summary: We introduce ReST-RL, a hierarchical reinforcement learning architecture that explicitly decouples locomotion from payload stabilization.<n>In simulation, the residual design significantly outperforms end-to-end baselines in gait smoothness and orientation accuracy.<n>This modular approach demonstrates highly reliable zero-shot sim-to-real generalization across various objects and external force disturbances.
• Score: 4.827821660801356
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Stabilizing unsecured payloads against the inherent oscillations of dynamic bipedal locomotion remains a critical engineering bottleneck for humanoids in unstructured environments. To solve this, we introduce ReST-RL, a hierarchical reinforcement learning architecture that explicitly decouples locomotion from payload stabilization, evaluated via the SteadyTray benchmark. Rather than relying on monolithic end-to-end learning, our framework integrates a robust base locomotion policy with a dynamic residual module engineered to actively cancel gait-induced perturbations at the end-effector. This architectural separation ensures steady tray transport without degrading the underlying bipedal stability. In simulation, the residual design significantly outperforms end-to-end baselines in gait smoothness and orientation accuracy, achieving a 96.9% success rate in variable velocity tracking and 74.5% robustness against external force disturbances. Successfully deployed on the Unitree G1 humanoid hardware, this modular approach demonstrates highly reliable zero-shot sim-to-real generalization across various objects and external force disturbances.

Related papers

• A Neural Network Based Teleoperation for Remote Controlled Vehicles [98.38733381483934]
  Direct teleoperation of vehicles faces critical technical bottlenecks.<n> communication latency and the operator's inability to physically perceive unmodeled environmental disturbances.<n>We propose a tailored unilateral teleoperation framework.
  arXiv  Detail & Related papers  (2026-08-11T01:51:08Z)
• Shift & Drift: A Zero-Shot Benchmark for Generalizable and Robust Autonomous Driving Motion Planning [60.888463449435484]
  We present Shift & Drift, a novel dual-track benchmark designed to rigorously stress-test motion planners across two critical axes of distribution shift.<n>We show that imitation learning methods achieve high scores in ID benchmarks, but suffer from persistent drift when subjected to temporally correlated actuation noise.<n>Our findings reveal an empirical trade-off between imitation fidelity and closed-loop resilience, providing the community with a rigorous benchmark to evaluate progress toward reliable deployment.
  arXiv  Detail & Related papers  (2026-07-08T18:23:48Z)
• ReactiveBFM: Reactive Closed-Loop Motion Planning Towards Universal Humanoid Whole-Body Control [54.267632784131415]
  ReactiveBFM is a real-time-loop planning-control framework for humanoids.<n>It mitigates fatal cumulative exposure bias via a scheduled prefix sampling curriculum.<n>It demonstrates unprecedented physical agility across a vast repertoire of text-conditioned closed-loop motions.
  arXiv  Detail & Related papers  (2026-06-29T14:27:27Z)
• Agile Fall Recovery for Quadrotors with Bidirectional Thrust via Reinforcement Learning [8.39070114446824]
  We present an RL-based framework for autonomous fall recovery of a quadrotor from arbitrary ground attitudes to stable hover using only lightweight onboard sensors.<n>We show that agile quadrotor fall recovery can be achieved without explicit state estimation using only limited and unreliable onboard sensing.
  arXiv  Detail & Related papers  (2026-06-15T10:15:10Z)
• Clipping Bottleneck: Stabilizing RLVR via Stochastic Recovery of Near-Boundary Signals [83.0127582612634]
  Near-boundary Rescue (NSR) is a minimal, plug-and-play modification that retains out-of-bound tokens to recover lost signals.<n>NSR substantially improves training stability and delivers consistent gains over strong baselines such as DAPO and GSPO.
  arXiv  Detail & Related papers  (2026-05-21T16:45:31Z)
• Rollback-Free Stable Brick Structures Generation [51.856925926010184]
  Existing approaches rely on external physical simulators during inference to perform rejection sampling and brick-by-brick rollbacks.<n>We propose a reinforcement learning paradigm that shifts physical validity enforcement from test-time correction to training-time policy optimization.<n> Experimental results demonstrate that our approach achieves state-of-the-art generation quality while accelerating inference speed by orders of magnitude.
  arXiv  Detail & Related papers  (2026-05-07T21:06:44Z)
• Learning Reactive Dexterous Grasping via Hierarchical Task-Space RL Planning and Joint-Space QP Control [50.28263951510334]
  We propose a hybrid hierarchical control framework for reactive dexterous grasping.<n>The proposed framework explicitly decouples high-level spatial intent from low-level joint execution.<n>We extensively validate the proposed framework through a rigorous simulation-to-reality pipeline.
  arXiv  Detail & Related papers  (2026-05-05T04:49:38Z)
• CycleRL: Sim-to-Real Deep Reinforcement Learning for Robust Autonomous Bicycle Control [10.350603824555408]
  CycleRL is the first sim-to-real deep reinforcement learning framework for robust autonomous bicycle control.<n>In simulation, CycleRL achieves considerable performance, including a 99.90% balance success rate, a low steering tracking error of 1.15, and a velocity tracking error of 0.18 m/s.
  arXiv  Detail & Related papers  (2026-03-16T09:17:51Z)
• ASTER: Attitude-aware Suspended-payload Quadrotor Traversal via Efficient Reinforcement Learning [6.13534373131836]
  This paper presents ASTER, a robust RL framework that achieves, to our knowledge, the first successful autonomous inverted flight for the cable-suspended system.<n>We show remarkable agility, precise attitude alignment, and robust zero-shot sim-to-real transfer across complex trajectories.
  arXiv  Detail & Related papers  (2026-03-11T12:40:29Z)
• Transformer-Based Reinforcement Learning for Autonomous Orbital Collision Avoidance in Partially Observable Environments [0.3093890460224435]
  We introduce a Transformer-based Reinforcement Learning framework for autonomous orbital collision avoidance.<n>A central contribution of this work is the use of transformer-based Partially Observable Markov Decision Process architecture.<n>This integration provides a foundation for training collision avoidance agents that can operate more reliably under imperfect monitoring environments.
  arXiv  Detail & Related papers  (2026-02-05T04:57:58Z)
• Robust and Generalized Humanoid Motion Tracking [17.58241987932198]
  Learning a general humanoid whole-body controller is challenging because practical reference motions can exhibit noise and inconsistencies after being transferred to the robot domain.<n>We propose a dynamics-conditioned command aggregation framework that uses a causal temporal encoder to summarize recent proprioception and a multi-head cross-attention command encoder to selectively aggregate a context window.<n>The proposed method is evaluated under diverse reference inputs and challenging motion regimes, demonstrating zero-shot transfer to unseen motions as well as robust sim-to-real transfer on a physical humanoid robot.
  arXiv  Detail & Related papers  (2026-01-30T15:27:43Z)
• SpanNorm: Reconciling Training Stability and Performance in Deep Transformers [55.100133502295996]
  We propose SpanNorm, a novel technique designed to resolve the dilemma by integrating the strengths of both paradigms.<n>We provide a theoretical analysis demonstrating that SpanNorm, combined with a principled scaling strategy, maintains bounded signal variance throughout the network.<n> Empirically, SpanNorm consistently outperforms standard normalization schemes in both dense and Mixture-of-Experts (MoE) scenarios.
  arXiv  Detail & Related papers  (2026-01-30T05:21:57Z)
• Efficiently Learning Robust Torque-based Locomotion Through Reinforcement with Model-Based Supervision [14.246416873754905]
  We propose a control framework that integrates model-based bipedal locomotion with residual reinforcement learning.<n>Our method demonstrates improved robustness and generalization across a range of randomized conditions.
  arXiv  Detail & Related papers  (2026-01-22T16:56:52Z)
• Dual-quaternion learning control for autonomous vehicle trajectory tracking with safety guarantees [39.146761527401424]
  We propose a learning-based trajectory tracking controller for autonomous robotic platforms whose motion can be described kinematically.<n>The controller is formulated in the dual quaternion framework and operates at the velocity level, assuming direct command of angular and linear velocities.
  arXiv  Detail & Related papers  (2026-01-06T15:30:02Z)
• Drift No More? Context Equilibria in Multi-Turn LLM Interactions [58.69551510148673]
  contexts drift is the gradual divergence of a model's outputs from goal-consistent behavior across turns.<n>Unlike single-turn errors, drift unfolds temporally and is poorly captured by static evaluation metrics.<n>We show that multi-turn drift can be understood as a controllable equilibrium phenomenon rather than as inevitable decay.
  arXiv  Detail & Related papers  (2025-10-09T04:48:49Z)
• Feed Two Birds with One Scone: Exploiting Function-Space Regularization for Both OOD Robustness and ID Fine-Tuning Performance [72.57668440744301]
  We propose a novel regularization that constrains the distance of fine-tuning and pre-trained model in the function space with simulated OOD samples.<n>Our approach could consistently improve both downstream task ID fine-tuning performance and OOD robustness across a variety of CLIP backbones.
  arXiv  Detail & Related papers  (2025-08-31T12:14:34Z)
• Forecasting Continuous Non-Conservative Dynamical Systems in SO(3) [51.510040541600176]
  We propose a novel approach to modeling the rotation of moving objects in computer vision.<n>Our approach is agnostic to energy and momentum conservation while being robust to input noise.<n>By learning to approximate object dynamics from noisy states during training, our model attains robust extrapolation capabilities in simulation and various real-world settings.
  arXiv  Detail & Related papers  (2025-08-11T09:03:10Z)
• RIFT: Group-Relative RL Fine-Tuning for Realistic and Controllable Traffic Simulation [13.319344167881383]
  We introduce a dual-stage AV-centric simulation framework that conducts imitation learning pre-training in a data-driven simulator.<n>We then learn fine-tuning in a physics-based simulator to enhance style-level controllability.<n>In the fine-tuning stage, we propose RIFT, a novel group-relative RL fine-tuning strategy.
  arXiv  Detail & Related papers  (2025-05-06T09:12:37Z)
• ReCoM: Realistic Co-Speech Motion Generation with Recurrent Embedded Transformer [58.49950218437718]
  We present ReCoM, an efficient framework for generating high-fidelity and generalizable human body motions synchronized with speech.<n>The core innovation lies in the Recurrent Embedded Transformer (RET), which integrates Dynamic Embedding Regularization (DER) into a Vision Transformer (ViT) core architecture.<n>To enhance model robustness, we incorporate the proposed DER strategy, which equips the model with dual capabilities of noise resistance and cross-domain generalization.
  arXiv  Detail & Related papers  (2025-03-27T16:39:40Z)
• Integrating DeepRL with Robust Low-Level Control in Robotic Manipulators for Non-Repetitive Reaching Tasks [0.24578723416255746]
  In robotics, contemporary strategies are learning-based, characterized by a complex black-box nature and a lack of interpretability. We propose integrating a collision-free trajectory planner based on deep reinforcement learning (DRL) with a novel auto-tuning low-level control strategy.
  arXiv  Detail & Related papers  (2024-02-04T15:54:03Z)
• The Risk of Federated Learning to Skew Fine-Tuning Features and Underperform Out-of-Distribution Robustness [50.52507648690234]
  Federated learning has the risk of skewing fine-tuning features and compromising the robustness of the model. We introduce three robustness indicators and conduct experiments across diverse robust datasets. Our approach markedly enhances the robustness across diverse scenarios, encompassing various parameter-efficient fine-tuning methods.
  arXiv  Detail & Related papers  (2024-01-25T09:18:51Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.