HERB: Human-augmented Efficient Reinforcement learning for Bin-packing

• URL: http://arxiv.org/abs/2504.16595v1
• Date: Wed, 23 Apr 2025 10:24:36 GMT
• Title: HERB: Human-augmented Efficient Reinforcement learning for Bin-packing
• Authors: Gojko Perovic, Nuno Ferreira Duarte, Atabak Dehban, Gonçalo Teixeira, Egidio Falotico, José Santos-Victor,
• Abstract summary: We propose HERB, a human-augmented RL framework for packing irregular objects.<n>We first leverage human demonstrations to learn the best sequence of objects to pack.<n>Next, we train a placement algorithm that uses visual information to determine the optimal object positioning inside a packing container.
• Score: 7.600912109332151
• License: http://creativecommons.org/licenses/by/4.0/
• Abstract: Packing objects efficiently is a fundamental problem in logistics, warehouse automation, and robotics. While traditional packing solutions focus on geometric optimization, packing irregular, 3D objects presents significant challenges due to variations in shape and stability. Reinforcement Learning~(RL) has gained popularity in robotic packing tasks, but training purely from simulation can be inefficient and computationally expensive. In this work, we propose HERB, a human-augmented RL framework for packing irregular objects. We first leverage human demonstrations to learn the best sequence of objects to pack, incorporating latent factors such as space optimization, stability, and object relationships that are difficult to model explicitly. Next, we train a placement algorithm that uses visual information to determine the optimal object positioning inside a packing container. Our approach is validated through extensive performance evaluations, analyzing both packing efficiency and latency. Finally, we demonstrate the real-world feasibility of our method on a robotic system. Experimental results show that our method outperforms geometric and purely RL-based approaches by leveraging human intuition, improving both packing robustness and adaptability. This work highlights the potential of combining human expertise-driven RL to tackle complex real-world packing challenges in robotic systems.

Related papers

• A Data-Centric Revisit of Pre-Trained Vision Models for Robot Learning [67.72413262980272]
  Pre-trained vision models (PVMs) are fundamental to modern robotics, yet their optimal configuration remains unclear. We develop SlotMIM, a method that induces object-centric representations by introducing a semantic bottleneck. Our approach achieves significant improvements over prior work in image recognition, scene understanding, and robot learning evaluations.
  arXiv  Detail & Related papers  (2025-03-10T06:18:31Z)
• Sim-to-Real Reinforcement Learning for Vision-Based Dexterous Manipulation on Humanoids [61.033745979145536]
  This work investigates the key challenges in applying reinforcement learning to solve a collection of contact-rich manipulation tasks on a humanoid embodiment. Our main contributions include an automated real-to-sim tuning module that brings the simulated environment closer to the real world. We show promising results on three humanoid dexterous manipulation tasks, with ablation studies on each technique.
  arXiv  Detail & Related papers  (2025-02-27T18:59:52Z)
• SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning [82.46975428739329]
  We develop a library containing a sample efficient off-policy deep RL method, together with methods for computing rewards and resetting the environment.<n>We find that our implementation can achieve very efficient learning, acquiring policies for PCB board assembly, cable routing, and object relocation.<n>These policies achieve perfect or near-perfect success rates, extreme robustness even under perturbations, and exhibit emergent robustness recovery and correction behaviors.
  arXiv  Detail & Related papers  (2024-01-29T10:01:10Z)
• REBEL: Reward Regularization-Based Approach for Robotic Reinforcement Learning from Human Feedback [61.54791065013767]
  A misalignment between the reward function and human preferences can lead to catastrophic outcomes in the real world.<n>Recent methods aim to mitigate misalignment by learning reward functions from human preferences.<n>We propose a novel concept of reward regularization within the robotic RLHF framework.
  arXiv  Detail & Related papers  (2023-12-22T04:56:37Z)
• Tactile Active Inference Reinforcement Learning for Efficient Robotic Manipulation Skill Acquisition [10.072992621244042]
  We propose a novel method for skill learning in robotic manipulation called Tactile Active Inference Reinforcement Learning (Tactile-AIRL) To enhance the performance of reinforcement learning (RL), we introduce active inference, which integrates model-based techniques and intrinsic curiosity into the RL process. We demonstrate that our method achieves significantly high training efficiency in non-prehensile objects pushing tasks.
  arXiv  Detail & Related papers  (2023-11-19T10:19:22Z)
• Stabilizing Contrastive RL: Techniques for Robotic Goal Reaching from Offline Data [101.43350024175157]
  Self-supervised learning has the potential to decrease the amount of human annotation and engineering effort required to learn control strategies. Our work builds on prior work showing that the reinforcement learning (RL) itself can be cast as a self-supervised problem. We demonstrate that a self-supervised RL algorithm based on contrastive learning can solve real-world, image-based robotic manipulation tasks.
  arXiv  Detail & Related papers  (2023-06-06T01:36:56Z)
• Learning Physically Realizable Skills for Online Packing of General 3D Shapes [41.27652080050046]
  We study the problem of learning online packing skills for irregular 3D shapes. The goal is to consecutively move a sequence of 3D objects with arbitrary shapes into a designated container. We take physical realizability into account, involving physics dynamics and constraints of a placement.
  arXiv  Detail & Related papers  (2022-12-05T08:23:39Z)
• SAGCI-System: Towards Sample-Efficient, Generalizable, Compositional, and Incremental Robot Learning [41.19148076789516]
  We introduce a systematic learning framework called SAGCI-system towards achieving the above four requirements. Our system first takes the raw point clouds gathered by the camera mounted on the robot's wrist as the inputs and produces initial modeling of the surrounding environment represented as a URDF. The robot then utilizes the interactive perception to interact with the environments to online verify and modify the URDF.
  arXiv  Detail & Related papers  (2021-11-29T16:53:49Z)
• Accelerating Robotic Reinforcement Learning via Parameterized Action Primitives [92.0321404272942]
  Reinforcement learning can be used to build general-purpose robotic systems. However, training RL agents to solve robotics tasks still remains challenging. In this work, we manually specify a library of robot action primitives (RAPS), parameterized with arguments that are learned by an RL policy. We find that our simple change to the action interface substantially improves both the learning efficiency and task performance.
  arXiv  Detail & Related papers  (2021-10-28T17:59:30Z)
• Beyond Pick-and-Place: Tackling Robotic Stacking of Diverse Shapes [29.49728031012592]
  We study the problem of robotic stacking with objects of complex geometry. We propose a challenging and diverse set of objects that was carefully designed to require strategies beyond a simple "pick-and-place" solution. Our method is a reinforcement learning (RL) approach combined with vision-based interactive policy distillation and simulation-to-reality transfer.
  arXiv  Detail & Related papers  (2021-10-12T17:46:06Z)

This list is automatically generated from the titles and abstracts of the papers in this site.

This site does not guarantee the quality of this site (including all information) and is not responsible for any consequences.