Deep Reinforcement Learning for Sim-to-Real Transfer in Humanoid Robot Baristas

Ziyuan Wang1, Yefan Lin1, Jiahang Zhang1, Leyu Zhao1, Xiaojun Hei1,
1Huazhong University of Science and Technology
Paper Video Code

Robot Baristas Deep Reinforcement Learning for Sim-to-Real Transfer

Abstract

In recent years, an increasing amount of research efforts have been invested in home-service humanoid robots. However, due to the complexity of home environments, the robustness of robot manipulation demands higher requirements in accuracy and promptness.

In this paper, we study the coffee-making application as an example. We proposed a reinforcement learning robot manipulation method with visual perception for filling-up the sim-to-real gap. We constructed a high-fidelity coffee making digital twin simulation environment.

In addition, we extracted the key points of robot hands using computer vision algorithms to achieve consistency between the simulation and the real scene. The domain randomization was applied to make the policy tolerant to the positional deviation of the coffee machine and coffee cups. We deployed the trained policy to the real robot in a zero-shot transfer.

Our experimental results demonstrated that the proposed method achieved over 90% success rate in end-to-end coffee making tasks on both simulation and real robots. The proposed method significantly outperforms other image-based or point cloud based methods during the sim-to-real process.

Architecture of the simulated and real robots

Interpolate start reference image.

Sim-to-real overall process

Interpolate start reference image.

Video

Button Pressing

The robot completes the coffee making task by pressing the coffee machine button, waiting for the coffee to be made, and then taking out the coffee cup.

Interpolate start reference image.

Coffee Cup Grasping

The robot waits the coffee to be made and grasp the coffee cup, take the cup out from the coffee machine and deliver it to the customer.

Interpolate start reference image.

BibTeX

@article{Wang2021robot,
  author    = {Wang, Ziyuan and Lin, Yefan and Zhang, Jiahang and Zhao, Leyu and Hei, Xiaojun},
  title     = {Deep Reinforcement Learning for Sim-to-Real Transfer in Humanoid Robot Baristas},
  journal   = {ROBIO},
  year      = {2024},
}